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EDUCATION RESEARCH: EXPLORING 
OPPORTUNITIES TO STRENGTHEN THE 
INSTITUTE OF EDUCATION SCIENCES 


Tuesday, September 10, 2013 
U.S. House of Representatives 
Committee on Education and the Workforce 
Washington, DC 


The committee met, pursuant to call, at 10:01 a.m., in room 
2175, Rayburn House Office Building, Hon. John Kline [chairman 
of the committee] presiding. 

Present: Representatives Kline, Petri, Foxx, Walberg, Salmon, 
Rokita, Miller, Scott, Hinojosa, Tierney, Holt, Grijalva, Polls, and 
Bonamici. 

Staff present: Katherine Bathgate, Deputy Press Secretary; 
James Bergeron, Director of Education and Human Services Policy; 
Heather Couri, Deputy Director of Education and Human Services 
Policy; Lindsay Fryer, Professional Staff Member; Rosemary 
Lahasky, Professional Staff Member; Nancy Locke, Chief Clerk; 
Krisann Pearce, General Counsel; Dan Shorts, Legislative Assist- 
ant; Nicole Sizemore, Deputy Press Secretary; Alex Sollberger, 
Communications Director; Alissa Strawcutter, Deputy Clerk; Brad 
Thomas, Senior Education Policy Advisor; Tylease Alii, Minority 
Clerk/Intern and Fellow Coordinator; Jeremy Ayers, Minority Edu- 
cation Policy Advisor; Kelly Broughan, Minority Education Policy 
Associate; Jamie Fasteau, Minority Director of Education Policy; 
Scott Groginsky, Minority Education Policy Advisor; Eunice Ikene, 
Minority Staff Assistant; Brian Levin, Minority Deputy Press Sec- 
retary/New Media Coordinator; and Megan O’Reilly, Minority Gen- 
eral Counsel. 

Chairman Kline. A quorum being present, the committee will 
come to order. 

Good morning. Welcome to our hearing to discuss opportunities 
to improve the Institute of Education Sciences. 

I would like to extend a special welcome to our witnesses, whose 
testimony will provide valuable insight into ways we can better en- 
sure parents, teachers, school leaders, and policymakers have ac- 
cess to the most relevant education research. 

Established by the Education Sciences Reform Act in 2002, the 
Institute of Education Sciences is responsible for gathering infor- 
mation on education progress, conducting research on educational 
practices in the nation’s schools, and examining the quality of fed- 
eral education programs and initiatives. The information collected 

( 1 ) 



2 


and disseminated by the institute helps schools identify and imple- 
ment successful education initiatives. 

Additionally, the data allows taxpayers and congressional leaders 
to keep tabs on the federal investment in education, which is espe- 
cially important in these times of fiscal restraint. The Education 
Sciences Reform Act has been due for reauthorization since 2008 
and traditionally moves right after the Elementary and Secondary 
Education Act. In July the House approved the committee’s legisla- 
tion to rewrite ESEA, known as the Student Success Act. 

The Education Sciences Reform Act presents another opportunity 
to help provide teachers and parents the tools necessary to raise 
the bar in our schools and I look forward to working with my col- 
leagues on both sides of the aisle to develop smart policies that will 
improve the law. 

To lay groundwork for the reauthorization, last year Ranking 
Member George Miller and I asked the Government Accountability 
Office to conduct a study on the effectiveness of the institute’s re- 
search. Though the final report has yet to be released, we have re- 
ceived a few preliminary findings that highlight areas for improve- 
ment. 

Eor example, GAO confirms the institute has greatly improved 
the quality of education research over the last decade but notes 
there is often a significant delay in disseminating key data and 
findings to education officials. As a result, the research is not al- 
ways immediately relayed to parents and school leaders, reducing 
its usefulness and relevancy. 

GAO also found the institute does not always properly evaluate 
the efficacy of its own programs and research arms, which could 
lead to unnecessary costs, confusion, and redundancies. Currently, 
the institute operates 10 regional labs and 12 research and devel- 
opment centers to conduct research, provide technical assistance, 
and distribute data. Meanwhile, the Department of Education oper- 
ates five content centers and 16 comprehensive centers that serve 
some of the same purposes. 

As we develop policies to strengthen the institute, we should con- 
sider streamlining the federal research structure to reduce duplica- 
tion, enhance accountability, and make it easier for states and 
school districts to access important information. We must also en- 
sure the Institute of Education Sciences has the flexibility nec- 
essary to modernize its research methods and keep up with new 
developments in education delivery and practice. Einally, we must 
acknowledge that the value of the institute’s research depends on 
its political autonomy and take the necessary steps to protect the 
organization’s independence. 

We are fortunate to have with us several witnesses who can help 
us better understand what is and is not working with the Institute 
of Education Sciences, including a representative from GAO who 
can provide more information on the aforementioned study. Their 
testimony will inform our efforts to reauthorize the Education 
Sciences Reform Act and help us craft policies that will improve 
the quality and usefulness of education research. 

With that, I now yield to my distinguished colleague, George Mil- 
ler, the senior Democratic member of the committee, for his open- 
ing remarks. 
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[The statement of Chairman Kline follows:] 

Prepared Statement of Hon. John Kline, Chairman, 

Committee on Education and the Workforce 

Established by the Education Sciences Reform Act in 2002, the Institute of Edu- 
cation Sciences is responsible for gathering information on education progress, con- 
ducting research on educational practices in the nation’s schools, and examining the 
quality of federal education programs and initiatives. 

The information collected and disseminated by the Institute helps schools identify 
and implement successful education initiatives. Additionally, the data allows teix- 
payers and congressional leaders to keep tabs on the federal investment in edu- 
cation, which is especially important in these times of fiscal restraint. 

The Education Sciences Reform Act has been due for reauthorization since 2008 
and traditionally moves right after the Elementary and Secondary Education Act. 
In July the House approved the committee’s legislation to rewrite ESEA, known as 
the Student Success Act. The Education Sciences Reform Act presents another op- 
portunity to help provide teachers and parents the tools necessary to raise the bar 
in our schools, and I look forward to working with my colleagues on both sides of 
the aisle to develop smart policies that will improve the law. 

To lay groundwork for the reauthorization, last year Ranking Member George 
Miller and I asked the Government Accountability Office to conduct a study on the 
effectiveness of the Institute’s research. Though the final report has yet to be re- 
leased, we have received a few preliminary findings that highlight areas for im- 
provement. 

For example, GAO confirms the Institute has greatly improved the quality of edu- 
cation research over the last decade, but notes there is often a significant delay in 
disseminating key data and findings to education officials. As a result, the research 
is not always immediately relayed to parents and school leaders, reducing its useful- 
ness and relevancy. 

GAO also found the Institute does not always properly evaluate the efficacy of its 
own programs and research arms, which could lead to unnecessary costs, confusion, 
and redundancies. Currently the Institute operates 10 regional labs and 12 research 
and development centers to conduct research, provide technical assistance, and dis- 
tribute data. Meanwhile, the Department of Education operates five content centers 
and 16 comprehensive centers that serve some of the same purposes. 

As we develop policies to strengthen the Institute, we should consider stream- 
lining the federal research structure to reduce duplication, enhance accountability, 
and make it easier for states and school districts to access important information. 
We must also ensure the Institute of Education Sciences has the flexibility nec- 
essary to modernize its research methods and keep up with new developments in 
education delivery and practice. Finally, we must acknowledge that the value of the 
Institute’s research depends on its political autonomy, and take the necessary steps 
to protect the organization’s independence. 

We are fortunate to have with us several witnesses who can help us better under- 
stand what is and is not working within the Institute of Education Sciences, includ- 
ing a representative from GAO who can provide more information on the aforemen- 
tioned study. Their testimony will inform our efforts to reauthorize the Education 
Sciences Reform Act and help us craft policies that will improve the quality and use- 
fulness of education research. 


Mr. Miller. Thank you very much, Mr. Chairman, for holding 
this important hearing. 

As we begin the new school year our nation’s schools stand on 
the threshold of major transitions. Schools are implementing new 
college and career ready standards, they are piloting new assess- 
ments aligned to those standards, and teachers and principals are 
being evaluated in new ways to measure and improve their effec- 
tiveness. Districts and states are implementing new accountability 
standards that focus intense efforts on turning around their most 
struggling schools. 

These are seismic changes. To ensure that children benefit, we 
need to support robust research to identify what is working and 
what can be improved. 
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When Congress passed the Education Sciences Reform Act in 
2002 we did so to complement the bipartisan effort of No Child Left 
Behind. We wanted to strengthen the quality and the rigor of edu- 
cation research and we wanted to take advantage of the new, rich 
data that would emerge from the NCLB requirements. 

Eleven years later, thanks to NCLB and ESRA, we have a 
wealth of data that can be used to measure what is working for 
students, make corrections where things are not working, and cre- 
ate ways to ensure continuous improvement in the future. That is 
the task I hope this committee will take as we review ESRA. 

In 2002 the quality of education research was lacking. Much of 
it was driven by politics rather than science. As a response, we cre- 
ated the Institute of Education Sciences. Its mission was to conduct 
scientifically rigorous research outside the influence of politics or 
trend of the moment. 

Since then, lES has crafted high standards for the research it 
funds. It trains and supports researchers across the country and 
the studies are peer reviewed. A decade later, education research 
is more rigorous and sound. 

However, research is not effective if it only answers abstract 
questions or only published in professional journals. Research must 
be relevant as well as rigorous. It must be widely shared with 
those who work with students in order to make a difference. 

I am pleased that lES has taken steps in that direction. In 2012 
lES overhauled the nation’s 10 regional research labs. It did so by 
connecting them with research alliances and policymakers and 
practitioners. The alliances work with the labs to identify pressing 
education problems in schools and then the labs develop and test 
strategies for solving them. 

Take New England, for example. The Northeastern regional lab 
created a research alliance on early childhood. That alliance gath- 
ered the region’s early childhood stakeholders to create a research 
agenda that focuses on standards, assessments, and practices to 
improve early childhood education. The resulting research will now 
be more useful to practitioners that have a ready-made network for 
disseminating it. 

As we examine ESRA and the role of lES I hope that we will 
keep this need for both rigor and relevance in mind. I hope that 
we will keep in mind that we in Congress must be good federal 
partners in this effort. That means we must provide stable and suf- 
ficient resources to lES to do its job. 

The lES budget in 2012 fell just short of $600 million. That is 
less than 1 percent of our overall federal education budget. Other 
fields invest far more in research and development. 

We should take a serious look at this. More money is not always 
the answer, but sufficient money is. 

Also, we must not forget that sequestration cuts this year are 
limiting research right now, and another round of cuts beginning 
in January could have a crippling impact in destabilizing ongoing 
and vital research. I hope that my colleagues will join me in seek- 
ing ways to invest in education research in a smart way that 
avoids waste but also is sound and avoids harmful austerity. 

I thank all of the witnesses for appearing today and I am pleased 
that there is such an interest and leadership in addressing the 
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quality of education research in America, and I look forward to all 
of your testimony. 

Thank you so much. 

[The statement of Mr. Miller follows:] 

Prepared Statement of Hon. George Miller, Senior Democratic Member, 
Committee on Education and the Workforce 

Chairman Kline, thank you for holding this important hearing. As we begin a new 
school year, our nation’s schools stand on the threshold of major transitions. 

Schools are implementing new college and career ready standards. And they are 
piloting new assessments aligned to those standards. Teachers and principals are 
being evaluated in new ways to measure and improve their effectiveness. Districts 
and states are implementing new accountability systems that focus intense efforts 
on turning around their most struggling schools. 

These are seismic changes. To ensure that children benefit, we need to support 
robust research to identify what is working and what can be improved. 

When Congress passed the Education Sciences Reform Act in 2002, we did so to 
complement the bipartisan effort of No Child Left Behind. We wanted to strengthen 
the quality and rigor of education research. And we wanted to take advantage of 
the new, rich data that would emerge from NCLB requirements. 

Eleven years later, thanks to NCLB and ESRA, we have a wealth of data that 
can be used to measure what is working for students, make corrections where things 
are not working, and create ways to ensure continuous improvement in the future. 
That is the task 1 hope this Committee will take up as we review ESRA. 

In 2002, the quality of education research was lacking. Much of it was driven by 
politics rather than science. As a response, we created the Institute of Education 
Sciences. Its mission was to conduct scientifically rigorous research, outside the in- 
fluence of politics or the trend of the moment. 

Since then, lES has crafted high standards for the research it funds. It trains and 
supports researchers across the country. Its studies are peer reviewed. A decade 
later education research is more rigorous and sound. 

However, research is not effective if it only answers abstract questions or is pub- 
lished in a professional journal. Research must be relevant as well as rigorous. And 
it must be widely shared with those who work with students, in order to make a 
difference. 

I am pleased that lES has taken some steps in this direction. In 2012 lES over- 
hauled the nation’s 10 regional research labs. It did so by connecting them with re- 
search alliances of policymakers and practitioners. The alliances work with the labs 
to identify pressing education problems in schools and then the labs develop and 
test strategies for solving them. 

Take New England, for example. The Northeastern regional lab created a re- 
search alliance on early childhood. That alliance gathered the region’s early child- 
hood stakeholders to create a research agenda that focuses on standards, assess- 
ments, and practices to improve early education. The resulting research will now be 
more useful to practitioners and have a ready-made network for disseminating it. 

As we examine ESRA and the role of lES, I hope we will keep this need for both 
rigor and relevance in mind. I hope we also keep in mind that we in Congress must 
be good federal partners in this effort. That means we must provide stable and suffi- 
cient resources to lES to do its job. 

The lES budget for FY12 fell just short of $600 million. That is less than 1 per- 
cent of our overall federal education budget. Other fields invest far more in research 
and development. We should take a serious look at this. More money is not always 
the answer, but sufficient money is. 

Also, we must not forget that sequestration cuts this year are limiting research 
right now. And another round of cuts beginning in January could have a crippling 
impact and destabilize ongoing and vital research. I hope my colleagues will join me 
in seeking ways to invest in education research in a smart way that avoids waste; 
but also in a sound way that avoids harmful austerity. 

I thank all the witnesses for appearing today. I am pleased that there is such in- 
terest and leadership in addressing the quality of education research in America. 

I look forward to your testimony. I yield back. 


Chairman Kline. Thank the gentleman. 

Pursuant to committee rule 7(c), all committee members will be 
permitted to submit written statements to be included in the per- 
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manent hearing record, and without objection the hearing record 
will remain open for 14 days to allow statements, questions for the 
record, and other extraneous material referenced during the hear- 
ing to he submitted in the official hearing record. 

It is now my pleasure to introduce our very distinguished panel 
of witnesses. 

First, Mr. George A. Scott is the director for education, work- 
force, and income security with GAO. He has previously testified 
before both the House and Senate on the agency’s work sur- 
rounding K-12 education and student financial aid programs. 

Dr. Bridget Terry Long is the Xander professor of education and 
economics and academic dean at the Harvard Graduate School of 
Education. She is also chair of the National Board for Education 
Sciences. 

Dr. James Kemple is executive director of the Research Alliance 
for New York City Schools at New York University. 

And Ms. Kathy Christie is the vice president for knowledge and 
information management and dissemination at the Education Com- 
mission of the States. 

Welcome, all of you. Before I recognize each of you to provide 
your testimony let me remind you of our very nifty lighting system 
here. 

You will each have 5 minutes to present your testimony. When 
you begin the light will turn green — no surprises here. When you 
have a minute left the light will turn yellow, and when your time 
is expired the light will turn red. 

And I am reluctant to gavel a witness down but I would ask you 
to try to wrap up at that point so that each of you has a chance 
to provide your testimony and we have a chance to engage in a dis- 
cussion. 

I will now like to recognize Mr. Scott for 5 minutes. 

Sir, you are recognized. 

STATEMENT OF GEORGE A. SCOTT, DIRECTOR FOR EDU- 
CATION, WORKFORCE, AND INCOME SECURITY ISSUES, U.S. 

GOVERNMENT ACCOUNTABILITY OFFICE 

Mr. Scott. Thank you. 

Chairman Kline, Ranking Member Miller, and members of the 
committee, thank you for inviting me here today to discuss the pre- 
liminary results of our review of the Department of Education’s In- 
stitute of Education Sciences, lES. 

The Education Sciences Reform Act outlines a broad mission for 
lES, including to expand the knowledge and understanding of edu- 
cation and to provide this information to a wide range of stake- 
holders. My testimony will focus on the extent to which lES sup- 
ports high quality research, disseminates relevant products to the 
education field, and coordinates within education and with other 
federal agencies. 

In summary, we found that since its creation lES has substan- 
tially improved education research. In 2007 the Office of Manage- 
ment and Budget concluded that lES had transformed the quality 
and rigor of research within the Department of Education. It had 
also increased the demand for scientifically based evidence of effec- 
tiveness in the education field as a whole. 
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Many stakeholders said that lES’s research standards improved 
the quality of and had a positive influence on education research. 

While lES has improved the quality of education research, its re- 
search is sometimes of limited usefulness to policymakers and prac- 
titioners. Some stakeholders told us that the evaluations supported 
by lES may not be completed soon enough to inform important pol- 
icy decisions. Eor example, officials from one organization said that 
lES’s evaluation of the Race to the Top and school improvement 
grant programs will not be released in time to give the states op- 
portunity to implement lessons learned from these studies before 
funding for these programs end. 

To address concerns about the relevance of its research lES is so- 
liciting feedback from stakeholders. Eor example, lES recently con- 
vened a group of state and local education officials to discuss the 
strengths and weaknesses of the regional education labs and the 
What Works Clearinghouse. 

Despite these efforts, lES does not have a systematic process for 
incorporating feedback from stakeholders into its research agenda. 
We consider having such a process to be a key element of pro- 
moting a sound federal research program. 

Eurther, lES cannot demonstrate the impact of its efforts to im- 
prove the quality and relevance of its research in some areas. lES’s 
performance measures no longer capture the full range of lES re- 
search and priorities. In some cases these measures are no longer 
relevant to managing the agency’s operations. 

Additionally, lES does not publicly report on the performance of 
the regional education labs, which constitute one of the agency’s 
largest investments. As we have previously reported, without ap- 
propriate performance measures agencies may be at risk for failing 
to achieve their stated goals. 

lES’s research and technical assistance group have taken various 
steps to provide relevant research. They have engaged policy- 
makers and practitioners in planning research and technical assist- 
ance activities. All three groups also use a range of methods to dis- 
seminate their products. 

Despite these efforts, stakeholders have raised concerns. Eor ex- 
ample, some stakeholders said that they do not find research by 
the regional education labs to be as relevant or as timely as other 
sources of information. Stakeholders also noted that further efforts 
are needed to better market the research from these groups so that 
they reach intended audiences. 

Einally, lES takes a number of steps to coordinate with other 
federal agencies to increase the use of research evidence in guiding 
funding decisions. lES also coordinates within Education to facili- 
tate collaboration among various research programs. 

However, despite these efforts, the Department of Education 
faces challenges in funding and prioritizing evaluations. According 
to Education officials, efforts to prioritize evaluation projects are 
hindered in part because of statutory requirements. 

According to these officials. Education lacks the authority to com- 
bine evaluation funds from programs across the department and 
then use them to evaluate any other program. As a result, some 
evaluations may not occur and high priority evaluations may be de- 
layed. 
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In conclusion, lES has made significant contributions to 
strengthening the rigor of education research. However, it could 
build on these efforts by continuing to improve its ability to provide 
timely and relevant information. 

In addition, with a systematic process for incorporating stake- 
holder needs and more comprehensive and up-to-date performance 
measures lES would position itself to more fully achieve its mis- 
sion. Also, the ability to prioritize evaluations is critical to helping 
the Congress make informed decisions about programs. As we com- 
plete our work we will consider any recommendations needed to ad- 
dress these issues. 

Mr. Chairman, this concludes my remarks. Thank you. 

[The statement of Mr. Scott may be accessed at the following 
Internet address:] 

http:! I www.gao.gov I assets 1 660 1 657669.pdf 


Chairman Kline. Thank you very much. 

Dr. Long, you are recognized. 

STATEMENT OF DR. BRIDGET TERRY LONG, XANDER PRO- 
FESSOR OF EDUCATION AND ECONOMICS, ACADEMIC DEAN, 

HARVARD GRADUATE SCHOOL OF EDUCATION, CHAIR, NA- 
TIONAL BOARD FOR EDUCATION SCIENCES, INSTITUTE OF 

EDUCATION SCIENCES 

Ms. Long. Thank you, and good morning. 

Chairman Kline and memb^ers of the committee, thank you for 
the opportunity to appear before you today. As noted, I am also 
chair of the National Board for Education Sciences. 

The board is independent of lES and we are tasked with advising 
the lES director and reviewing and evaluating its work. In this 
way, we provide a critical but also constructive perspective on the 
activities of lES. 

My testimony reflects the discussions and recommendations by 
the board as well as my observations as an experienced researcher. 

As we work to raise student achievement, foster productive 
learning environments, and bolster the social contributions of our 
schools and universities, the knowledge, inventions, and partner- 
ships created through education research are essential. It is 
through research that we determine the best ways to produce the 
needed gains and help to make tough decisions about how to use 
our limited funds. 

During the short history of lES it has filled an essential role in 
providing and encouraging the necessary conditions for high qual- 
ity research in education. I focused my comments on three main 
contributions. 

Eirst, lES has taken the role of creating a series of important 
public goods. By “public goods” I mean things that benefit us all, 
but many of these goods would not have been produced without 
government intervention. 

As noted by Chairman Kline, lES provides the foundations of fac- 
tual information and research with the collection of clear, con- 
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sistent, high quality data. Additionally, lES serves as a repository 
and distribution center of objective research. 

This is vitally important because the education space is filled 
with many organizations, companies, and individuals who have 
varying objectives, agendas, and degrees of expertise. Therefore, it 
can be difficult to discern between the many studies, reports, and 
assertions of what is fact versus what is fiction, and lES stands as 
the best authority of rigorous research free from influence. 

Second, lES has led the way in efforts to reevaluate and redefine 
the standard of what is considered the best evidence. Before the 
creation of lES many lamented that education research was failing 
to answer important educational questions in convincing ways. One 
example of what lES has done is push for randomized control 
trials, which is a gold standard in research and often used in the 
field of medicine. 

By providing support and encouraging researchers to develop 
ways of conducting RCTs while still being sensitive to the needs of 
students, education research has progressed in fundamental ways 
with new, important evidence on the effects of key programs and 
interventions. Moreover, lES continues to engage the field in con- 
versations about rigor, as demonstrated by technical work groups 
tasked with ensuring that evaluations provide unbiased assess- 
ments. 

Third, lES has influenced the kind of research that is done. 
While there are many organizations that conduct education re- 
search, most focus on only a handful of topics and are only able to 
do projects of limited size. But education is all encompassing, from 
the wide array of the types of students, environments, needs, and 
goals, and there is much work to be done. 

With a national platform, lES has the unique ability to leverage 
researcher and practitioner expertise by signaling and providing in- 
centives to conduct studies on issues of importance for the country. 
This includes large-scale projects that would not otherwise be con- 
ducted but shed an considerable light on important issues. 

Another contribution has been to emphasize the importance of 
partnerships with researchers and schools, districts, and edu- 
cational agencies. By working closely with the field, researchers are 
much more likely to produce research that is relevant and useful 
in practice. 

Einally, it is important to note that lES has been instrumental 
in attracting talent to the study of education with training, tools, 
and resources to support high quality research. 

While the accomplishments of lES are numerous, the board and 
lES are committed to continuous improvement. As noted earlier, 
lES has filled a gap for the nation by providing clear, objective in- 
formation. However, more could be done to communicate and dis- 
seminate this information. 

This is a challenging feat. Education has an incredibly large 
range of stakeholders and multiple audiences to address, including 
policymakers, practitioners — from teachers, to superintendents, to 
state agencies — researchers, and students and their families. Each 
group needs different kinds of information in different forms. 
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There are many examples of success and promise. For example, 
the practice guides distill a wealth of research into clear steps for 
teachers to take to improve the learning of students. 

Additionally, there have been many efforts to improve the work 
of lES. They include revisions to the Web site, establishing a grant 
competition to create a research and development center on knowl- 
edge utilization. To ensure reliance and usability lES has also re- 
vised and renegotiated the contracts for the RELs and just last 
month convened a product feedback and development meeting with 
stakeholders from across the country. 

I have also entered into the record a full list of the board’s rec- 
ommendations regarding ESRA. 

[The information follows:] 

THE NATIONAL BOARD OF EDUCATION SCIENCES (NBES) 

Advisory Board to the Director of the Institute of Education Sciences (lES), 

U.S. Department of Education Dr. Bridget Terry Long, Chair 

Recommendations for the Rcauthorization of the 
Education Sciences Reform Act (ESRA) 

At the June 20, 2012 NBES meeting, Board members discussed recommendations 
to revise ESRA. These recommendations build from previous suggestions made by 
the NBES in May 2008 with several additional changes and revisions. The rec- 
ommendations fall into three categories: 

• Definitional changes 

• Substantive changes in the Institute or Board’s functioning or powers 

• Administrative or “housekeeping” changes to the bill 

DEFINITIONAL CHANGES 

1. Definitions related to “Scientific Research,” etc. 

On pages 4-5, we recommend changes related to definitions of “scientific re- 
search.” Congress has moved towards defining principles of scientific research rath- 
er than defining scientifically-based research. As noted on page 5, the NBES agrees 
with the Department of Education’s position that any definition of “scientific re- 
search,” etc. in ESRA should be consistent with the definitions used in other bills 
such as the Elementary and Secondary Education Act. 

2. Changes in IBS’s mission 

On page 7, the NBES recommends modifying the initial lines of lES’s mission to 
read: 

The mission of the Institute is to provide national leadership in expanding reliable 
evidence on which to ground education practice and policy and to encourage its use 
by parents, educators, students, researchers, policymakers, and the general public 

^ ^ 5k 

The major differences between our recommendation and the original are to re- 
place “expanding fundamental knowledge” with “expanding reliable evidence” and 
to add the words “encourage its use”. The new definition, which focuses on providing 
evidence and encouraging its use, is much closer to what lES does and is more ob- 
jective than the existing definition’s focus on fundamental knowledge and under- 
standing. 

Note: This is not marked on the attached draft. 

SUBSTANTIVE CHANGES RELATED TO POWERS, FUNCTIONS, AND TERMS OF OFFICE 

1. Delegation of “Other Activities” 

Changes on page 9 would leave it at the discretion of the lES Director to accept 
additional assignments from the Secretary of Education if they were consistent with 
lES’s mission and priorities. The existing language gives the Secretary the power 
to simply assign such activities to lES. 

2. Provisions related to the IBS Director 

Page 9 removes language which is no longer relevant related to the appointment 
of the first lES Director. In addition, language is added making it possible for a Di- 
rector to be nominated for a second term, and for a sitting Director to serve up to 
an additional year if his/her successor has not been appointed. 
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Additional new language on page 9-10 makes the lES Director eligible for “critical 
pay” under the Federal Workforce Flexibility Act of 2004. The explanatory note in 
the mark-up states, “Many people who might be qualified to be Director are unwill- 
ing to do so over a 6 year term at the rate of pay [specified in existing law]. This 
addition provides pay flexibility in recruiting a director, and would be subject to the 
recommendation of the Board.” Page 13 adds the language related to the Board’s 
ability to make recommendations in this area. 

New language on page 10 specifies that the IFS Director reports directly to the 
Secretary of Education. The explanatory note in the mark-up states, “A direct re- 
porting line to the Secretary is important to maintaining the status and inde- 
pendent functioning of IFS within the Department.” 

3. Requirement that lES Director submit a biennial plan of activities to the Board 

for advice 

Page 10 adds new language to the Director’s duties requiring him/her to submit 
a biennial plan of activities to the Board every two years. New language on page 
12 adds reviewing and advising the Director on the plan of activities to the Board’s 
duties. (Note that the Board’s approval of the plan is not required.) 

4. NBES: Organizations that advise the President on Board members 

Existing language in ESRA requires the President to solicit advice regarding indi- 
viduals to serve on the Board from the National Academy of Sciences, the National 
Science Board, and the National Science Advisor. New language on page 13 would 
add the Board itself, the American Educational Research Association, the Society for 
Research on Educational Effectiveness, and the National Academy of Education to 
the list of groups that the President must solicit advice from. 

5. NBES: Board terms 

Page 14 includes a number of changes aimed at fixing some of the difficulties that 
the Board has consistently encountered since its founding, including numerous va- 
cancies and attenuated terms. 

Page 14 adds language specifying that a member’s 4-year term commences from 
the date of their appointment. Current practice has been to appoint members to 4- 
year slots, whose beginning and end dates are calculated based on three cohorts tied 
to the original legislation. That is: there are five Board slots tied to an initial term 
expiration date of 11/28/08; five Board slots tied to an initial term expiration date 
of 11/28/06; and five Board slots tied to an initial term expiration date of 11/28/07. 
If a Board member was nominated or confirmed to a slot that was close to its expi- 
ration date, their effective term dating from their actual appointment might be as 
short as a year. Page 14 also strikes language which is no longer relevant related 
to the initial appointment of Board members to staggered terms. 

Page 14 also adds a new provision (similar to the provision for the lES Director) 
allowing Board members to serve up to an additional year after their term has ex- 
pired if their successor has not been appointed. 

The net effect of the changes listed on page 14 will be to greatly reduce the num- 
ber of unfilled vacancies on the Board and to make it much more likely that at all 
times the Board will have close to its full complement of 15 members that ESRA 
stipulates. 

6. NBES: Executive Director 

New language on page 15 of the mark-up provides greater detail regarding the 
Executive Director position. Board members favored revising ESRA to give the 
Board hiring and evaluation authority over the NBES Executive Director. 

7. NBES: Charitable contributions 

New language on page 16 would allow the Board to accept charitable donations 
to further the mission of the Board. This would allow the Iloard to provide coffee 
during advisory board meetings. 

8. NBES: Standing committee structure 

Pages 16 strikes language related to NBES’s standing committee structure. The 
existing language specifies a standing Board committee corresponding to each na- 
tional education center (e.g., NCES, NCER). The Board has never functioned this 
way in practice. This is due, in part, to the fact that at times Board membership 
has dwindled to as few as 6 members. 

In place of the struck language, the mark-up adds new permissive language on 
page 16 that allows the Board to establish standing committees related to the 
Board’s responsibilities. 
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9. Commissioner’s pay 

Similar to the new language related to the lES Director’s pay, page 18 adds lan- 
guage allowing Commissioners to be eligible for critical pay under the provisions of 
Federal Workforce Flexibility Act of 2004. The rationale is similar to that for the 
IFS Director: to enhance recruitment flexibility for Commissioners. 

10. The appointment process for the NCES Commissioner 

Page 18 has existing language that has the NCFS Commissioner be appointed by 
the President and confirmed by the Senate. The Board made no recommendation re- 
garding changing to the appointment process for the NCFS Commissioner, because 
opinion was evenly divided among the members. Some felt strongly that the current 
requirement that the NCFS Commissioner be appointed by the President and con- 
firmed by the Senate reflects a hard-won acknowledgment that education statistics 
deserve national-level prestige. Others instead felt that the procedure for appointing 
the NCFS Commissioner should be the same as that for the other IFS Center com- 
missioners to support smooth and efficient functioning of IFS. 

11. National Research and Development Centers 

Pages 21-22 strike existing language related to National Research and Develop- 
ment Centers and replaces it with a section titled, “Priorities for Long-Term Re- 
search Activities.” The changes remove language requiring the funding of at least 
eight National R&D Centers as well as requirements related to the topics assigned 
to the centers. The replacement language does not address the number of centers 
funded and allows the NCFR Commissioner to choose topics consistent with IFS’s 
priorities. The feeling is that the NCFR commissioner and the director, with the 
counsel of the Board, should be able to determine the best funding mechanisms and 
funding levels for advancing IBS’s long-term research priorities rather than having 
Congress earmark particular centers and levels of funding. 

12. Removal of privacy protection for individual school information 

Page 41 strikes language giving privacy protection to individually identifiable in- 
formation with respect to individual schools. The explanatory note in the mark-up 
states that, “Schools do not receive privacy protection elsewhere in federal statute 
or regulations. Many lES reports from NCFS require that schools be identified, e.g., 
the Common Core of Data. The prohibition on revealing school identity means that 
useful information must be omitted from evaluation reports. There is no compelling 
reason to maintain this protection for schools.” 

13. Adjustment to the circumstances under which the Director or Board members 
may be removed 

Pages 45-46 add language that allows the President to remove the Director and 
any Board member for cause, although the President must inform the Board of the 
cause for which the appointee is being removed. The original language did not in- 
clude the words “for cause,” nor was Board notification required. The original lan- 
guage also included the Commissioner for Education Statistics in these provisions. 
However, the mark-up’s proposed change to make the Commissioner of Education 
Statistics a Director-appointed position, like the other Commissioners, means that 
the Commissioner should be struck from the provisions of this section. 

14. Expansion of authorization related to data bases to be included in the statewide 
longitudinal data systems 

Page 52 adds language specifying the Higher Education Act and IDEA with re- 
gard to the development of statewide longitudinal data systems. The existing lan- 
guage only specifies the Elementary and Secondary Education Act. The change ex- 
pands the authority so that a broader range of educational records can be incor- 
porated into the supported data systems. 

15. NAEP reports 

Pages 55 and 57 add language related to authority over the content and release 
of National Assessment of Educational Progress (NAEP) reports. The mark-up’s ex- 
planatory note states, “There are ongoing disputes between NCES and the National 
Assessment Governing Board (NAGB) on the content and formatting of NAEP re- 
ports * * * These are lES/NCES reports published under the authority of the Direc- 
tor and Commissioner. The NCES commissioner needs to retain responsibility for 
the content of the reports.” And, “NAGB has taken the position that the NCES com- 
missioner’s role at the release event is entirely at the discretion of NAGB. Because 
the findings being released are from an NCES statistical report, the commissioner 
or his delegate should be responsible for presenting the findings.” 



13 


administeative/housekeeping changes 

1. Delegation of authority 

Page 8 contain new language aimed at delineating the delegation of authority be- 
tween the lES Director and the Secretary of Education. 

2. Removal of language related to the role of the National Assessment Governing 

Board (NAGB) 

Four provisions related to NAGB are struck on pages 8-9. The rationale given was 
that a preceding paragraph clearly articulated NAGB’s role and so the subsequent 
four provisions were redundant. 

3. Other changes in the Direetor’s duties 

Page 10 changes language related to peer review from “establish” to “maintain,” 
reflecting the fact that peer review procedures have already been established. 

Another change on page 11 specifies that the Director will coordinate with the 
Secretary of Education to insure that lES’s findings are used by all of ED’s technical 
assistance providers, and not just the 15 comprehensive assistance centers. 

4. Review of publications, not “products” 

Page 11 strikes the term “products” from the section pertaining to the Director’s 
review of evidence-based claims in ED publications. The rationale is that lES cannot 
review products, only publications that make scientific claims. 

5. Requirement that lES priorities be proposed every 6 years 

New language on page 11 requires the lES Director to submit priorities for the 
Institute to the Board for approval at least every 6 years. This would put into stat- 
ute what has occurred in practice. 

6. Peer review standards and NCER 

Page 19 strikes language requiring the National Center for Education Research 
(NCER) to maintain peer review standards. This would conform to lES’s actual 
practice, which is to have the Scientific Review Office maintain lES’s standards re- 
lated to peer review. 

7. Replaeing ‘Commissioner’ with ‘Center’ 

Pages 26 and 29 replace ‘Commissioner’ with ‘Center’ for the sake of consistency. 

8. Removal of outdated language related to NCEE 

Page 32 strikes language regarding the award of specific contracts which is out- 
dated and not relevant for reauthorization. Similarly, pages 34-35 strikes additional 
outdated language that is not relevant to reauthorization. 


[An additional submission, the document, “NBES Markup, June 
2012, Education Sciences Reform, Public Law 107-279,” may be 
accessed at the following Internet address:] 

http: 1 1 ies.ed.gov I director I hoard I pdf I NBESmarkupESRA.pdf 


Ms. Long. So in summary, lES has made many contributions, 
though there is still work to be done, and we look forward to this 
discussion. 

Thank you. 

[The statement of Dr. Long follows:] 

Prepared Statement of Dr. Bridget Terry Long, Ph.D., Xander Professor of 
Edueation and Economies Academic Dean, Harvard Graduate School of 
Education; Chair, National Board for Education Sciences 

Mr. Chairman and members of the committee, thank you for the opportunity to 
appear before you today. 

My name is Dr. Bridget Terry Long, and I am the Academic Dean and Xander 
Professor of Education and Economics at the Harvard Graduate School of Education. 
Beyond my expertise as a researcher and faculty member, I am also the Chair of 
the National Board for Education Sciences, the advisory board of the Institute of 
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Education Sciences (lES). The National Board for Education Sciences is independent 
of lES, and we are tasked with advising the Director and reviewing and evaluating 
the work of lES. In this way, we provide a critical but also constructive perspective 
on the activities of lES. 

My testimony reflects discussions and recommendations made by the Board as 
well as my observations as an experienced educational researcher who has 
interacted with lES on many levels. My comments today aim to provide an objective 
assessment of the role of lES, its contributions, and areas for improvement. 

The Role of IE S 

In our current environment, educational research has become even more impor- 
tant as the penalties of poor achievement and lack of opportunity have never been 
greater. As we work to raise student achievement, foster productive learning envi- 
ronments, and bolster the social contributions of our schools and universities, the 
knowledge, inventions, and partnerships created through educational research are 
essential — it is through research that we determine the best ways to produce the 
needed gains and help to make tough decisions about how to use our limited funds. 
Before we can debate what policies we should implement, we first need a clear un- 
derstanding of the facts and to have an accurate sense of the real costs and benefits 
of any policy or program. In essence, research is the foundation for improving edu- 
cation. 

During the short history of lES, it has filled an essential role in providing and 
encouraging the necessary conditions for high-quality education research. While its 
impact is evident in many ways, I focus my comments on three main contributions. 
First, lES has taken the role of creating a series of public goods that no one else 
would or could do without concerns about possible bias. Second, it has led the way 
in efforts to reevaluate and redefine the standard of what is considered good evi- 
dence. Third, lES has influenced the kind of educational research that is done by 
making possible large-scale studies, pushing researchers to work closely with practi- 
tioners to ensure relevance and usability, and holding an unwavering focus on serv- 
ing the national good. 

(1) Creating Necessary Public Goods 

As a federal entity, lES has taken leadership to provide several key public goods 
needed to support a strong educational system and research. By public goods, I 
mean things that benefit us all, but many of these goods would not otherwise be 
produced without government intervention. For example, lES provides the founda- 
tions of factual information and research with the collection of clear, consistent, 
high-quality data through the National Center for 

Education Statistics (NCES).^ It is through the efforts of lES, which conducts its 
work free from political influence, that we are able to understand trends in our stu- 
dent populations, schools and universities, and an array of inputs and outcomes that 
span early childhood to adult education. These data also make possible a wealth of 
research conducted on every aspect of education. 

Additionally, lES serves as a repository and distribution center of research, both 
studies funded by lES and those that are not. The dissemination and communica- 
tion of objective information is a critical one for the nation. The education space is 
filled with many organizations, companies, and individuals who have varying objec- 
tives, agendas, and degrees of expertise. Therefore, it can be difficult to sort between 
the many studies, reports, and assertions to determine what is fact versus what is 
fiction. Moreover, the research community often lacks the training and incentives 
to translate complex research for a lay audience. In such a crowded space, lES 
stands as the best authority of rigorous research free from influence. It has helped 
to clarify what is known about issues related to large educational debates. More- 
over, it has been helpful in discerning between conflicting and confusing reports on 
important issues. It has used its convening power to bring together researchers from 
various backgrounds to discuss the issues and coordinate research.^ It has also con- 
ducted evaluations of federal initiatives.^ 

(2) Setting the Standards of “Good Evidence” 

Before the creation of lES, many lamented that educational research was failing 
to answer important questions in convincing ways. The varying quality of research 
and lack of attention to certain issues led some to dismiss the educational research 
base as inadequate. lES has changed this dramatically by leading a critical assess- 
ment of past research and initiating a number of debates about what are appro- 
priate methods and standards of rigor for the different approaches to educational 
research. 

One concrete example of this has been the push for randomized controlled trials 
(RCTs), which are considered the gold-standard of research and often used in the 
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field of medicine. Prior to IBS’s leadership, RCTs were rarely conducted in education 
and not valued among many researchers. However, by pushing the field, providing 
support, and engaging researchers to develop ways of conducting such analyses 
while still being sensitive to needs of students and practitioners, educational re- 
search has progressed in fundamental ways with new important evidence on the ef- 
fects of key programs and interventions. For example, in my own work with several 
colleagues, which was partially funded by IBS, we demonstrated that providing low- 
and moderate-income families with streamlined personal assistance to complete the 
federal college financial aid application had large effects on college attendance and 
persistence. Because we used a randomized controlled trial design, we were able to 
establish convincingly that our intervention was not only the cause of the edu- 
cational gains; importantly, the program was also inexpensive.'^ 

IBS continues to engage the field in conversations about rigor in educational re- 
search. This is demonstrated by technical working groups that are establishing 
standards for specific research methodologies and helping to ensure that evaluations 
provide unbiased and causally-valid assessments.® It is also worth noting that IBS 
has developed a rigorous peer review process for evaluating grant proposals. 

(3) Encouraging Relevant, Rigorous Researeh for the National Good 

IBS has used its resources and convening power to focus the field on research that 
is both rigorous and focused on shedding light on the major problems facing the 
country. By setting priorities and crafting calls for research proposals (i.e.. Requests 
for Proposals or RFPs), IBS has sent signals to the field about important topics that 
need answers, rigorous standards that must be upheld, and the importance of con- 
ducting research in partnership with practitioners. 

Additionally, it has made possible research studies that would not have otherwise 
been conducted. 

While there are private foundations and other organizations that support edu- 
cational research, most focus on only a handful of topics and fund projects of limited 
size. But education is all encompassing, from the wide array of types of students, 
environments, needs, and goals, and there is much work to be done. With a national 
platform, IBS has the unique ability to leverage researcher and practitioner exper- 
tise by signaling and providing incentives to conduct studies on issues of importance 
for the country. One way it has done this is by designing research competitions that 
focus on the major issues and areas of education. Along with this has come IBS’s 
emphasis on the importance of external validity in research, meaning that it has 
called for researchers to be accountable to external audiences on how the findings 
for one set of schools might be applicable to another set of schools. 

IBS has also been able to support large-scale projects that could not be easily 
funded by others. To learn more certain issues, studies must be large in scale and 
compare the experiences of districts across states or large populations of students. 
Without support from IBS, this type of work would often not be possible, and the 
knowledge base that is being built as a result of this work has been valuable in im- 
proving student outcomes. Taken together, IBS has both insured research on a 
breadth of topics while also making possible large-scale studies that have been in- 
credibly beneficial to our understanding of how to help students. 

Another way IBS has influenced the research community is by highlighting the 
importance of partnerships between researchers and schools, districts, or state edu- 
cational agencies. Because the delivery of education is the result of many actors, re- 
search can often be improved by being designed and conducted while working with 
practitioners. Additionally, by working closely with the field, researchers are much 
more likely to produce research that is relevant and useful in practice. However, 
such work can be difficult to manage and implement. 

IBS has pushed and supported such connections to the benefit of the research 
being conducted.® 

Finally, it is important to note that IBS has been instrumental in attracting tal- 
ent to tbe study of education. With the signals it sends about important issues in 
education and the support it gives for research, IBS has helped to attract a growing 
number of researchers with the tools and resources to support high-quality research 
and partner with the field. IBS is helping to produce the next generation of scholars 
and innovators who will help to solve important problems in education. 

The Strengths, Challenges, and Continuous Improvement oflES 

The accomplishments of IBS are numerous, and the researchers and innovators 
supported by IBS funding will continue to have positive impacts on the lives of stu- 
dents as well as many other parts of our society. Nevertheless, in light of the Board 
and IBS’s commitment to continuous improvement, it is clear more can and needs 
to be done. In this spirit, the Board has worked to advise, review, and advance the 
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activities of lES. The Board has matured to he an important place of feedback and 
expertise, and my comments here reflect continuing discussions between the Board 
and lES staff about how to address challenges facing the organization. 

As I noted earlier, the dissemination and communication role of lES is an impor- 
tant one. 

lES has filled a gap for the nation by providing clear, objective information and 
making it available to the public. While lES is a strong producer and supporter of 
information of value, it is still building capacity and expertise on how to disseminate 
that information, including methods that use the latest technology and outreach 
methods. This is a challenging feat. Unlike many other fields, education has large 
range of stakeholders and multiple audiences to address, including policymakers; 
practitioners from teachers to superintendents to state agencies; researchers; and 
students and their families. Each group needs different kinds of information in dif- 
ferent forms. 

The Board and lES staff believe strongly in the dissemination role of lES, and 
we have held a number of discussions on how to improve efforts. There are many 
examples of success and promise. For example, the Practice Guides distill a wealth 
of research into clear steps teachers can take to improve the learning of their stu- 
dents.^ The What Works Clearinghouse was created with the idea of helping the 
public understand research results and whether they were completed using rigorous 
methods. The dissemination of recent data reports and grant competitions include 
webinars and video media.® 

However, more could be done in terms of reaching out to the many audiences of 
educational data and research, and there are many efforts underway at lES to ad- 
dress this challenge. They include: 

• Revisions to the website to make it easier to find important research and facts. 
For instance, a new contract was awarded this year to manage and enhance the 
What Works Clearinghouse.® Additionally, as part of the REP for the Education Re- 
sources Information Center (ERIC), the contractor is expected to redesign the lES 
website to improve search capabilities and provide basic orientations to key topics 
and references for relatively inexperienced users. 

• lES added new requirements to research grant competitions for researchers to 
develop dissemination plans for their studies. Moreover, NCSER released a report 
on how to make research more understandable, and it was presented to its grant 
recipients.i'^ 

• Establishing a grant competition to create a Research and Development Center 
on Knowledge Utilization. This Center will explore questions of how education re- 
searchers can make their work more relevant and useful to practitioners located in 
state and local education agencies and in individual schools. This work is meant to 
address concerns that often there is only limited adoption of evidence-based prac- 
tices. 

Related to the issue of dissemination is the relevance and usability of the research 
produced and funded by lES. This has been a major focus of lES, and there are 
many instances of the Institute meeting this goal. As noted above, the growing at- 
tention to the importance of partnerships has broadened the number of studies done 
in concert with schools and districts, and this approach increases the likelihood that 
the results will be relevant and useful for practitioners. Still, this has been an area 
of constant reevaluation, and there have been many activities recently to improve 
this function of lES. For example: 

• Revising and renegotiating the contracts for the Regional Education Labs 
(RELs). For example, earlier this year, lES released revised criteria for REL pro- 
posals and products. The criteria focus on issues related to the technical rigor of 
products (e.g., data quality, analysis methods), the relevance of the work (i.e., 
whether it provides evidence that can inform a practitioner’s action or decision), and 
the readability of the products (i.e., whether the information is clear for its intended 
audiences). NCEE has also been working to build the capacity of the REL program 
by conducting webinars to help the RELs meet increasing standards in writing, col- 
laboration, and measurement. 

• Just recently, on August 12, 2013, lES convened a Product Feedback and Devel- 
opment Meeting with stakeholders from across the country to get suggestions about 
how to improve the usability and relevance of the products and services of the WWC 
and RELs.i’^ 

As an independent body tasked with providing constructive feedback to lES, the 
Board has been pleased with the fact that our feedback and that of others on these 
issues has been incorporated into the work of the Institute, and we believe these 
activities will help to strengthen lES’s impact. 

Another challenge facing lES is balancing the need to work in many areas with 
the reality of having limited resources. Because it is important to understand so 
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many facets of education and the populations it impacts, it can be difficult to 
prioritize some areas over others or to decide not to fund research in some areas 
at all. Touch choices sometimes have to be made. For instance, this year, lES will 
not hold research competitions in special education, However, lES is not taking 
a haphazard approach to this dilemma. Recent discussions between the Board and 
lES staff have concerned if and how the Institute might decide to prioritize funding 
decisions. Moreover, lES is attempting to understand and improve the impact of the 
overall portfolio of research supported with lES funding. Together, we have been ex- 
amining the research portfolios of NCER and NCSER to understand how lES might 
better target its research funding. 

Revising ESRA: Recommendations from the NBES 

At the June 20, 2012 NBES meeting. Board members discussed specific rec- 
ommendations to revise ESRA. These recommendations build from previous sugges- 
tions made by the Board in May 2008 with several additional changes and revisions. 
Most notably, we suggest: 

• Establishing a requirement that the lES Director submit a biennial plan of ac- 
tivities to the Board for advice. Currently, the lES Director is only required to sub- 
mit his or her priorities to the Board every six years. Although the Board has many 
informal opportunities to provide feedback to the Director based on the strong work- 
ing relationship between the current Board and current Director, the expectation of 
more frequent formal feedback should be documented. 

• Changing the term of a Board member to commence from the date of confirma- 
tion so that members have a full four years of service 

• Automatically extending by one year the terms of Board members whose succes- 
sors have not yet been appointed; this would help to ensure that the Board always 
has a sufficient number of members to be effective 

• Giving the Board hiring and evaluation authority over the NBES Executive Di- 
rector to ensure this role is independent of lES given the assessment duties of the 
Board 

• Allowing for flexibility in the pay of the lES Director and Commissioners by 
making these positions eligible for “critical pay” under the Federal Workforce Flexi- 
bility Act of 2004 

• Removing privacy protection for individual schools in data reports, a protection 
that does not exist in any other federal statue or regulation. The current prohibition 
on revealing school identity means that useful information must be omitted from 
evaluation reports. 

A full list of our recommendations and a marked-up copy of ESRA have been en- 
tered into the official record. 

Conclusions 

In summary, to have an informed populace and clarity on how best to educate our 
children and ourselves, there must be a robust foundation of high-quality data, rig- 
orous, objective research and strong communication of evidence on what works and 
what does not. It is clear that lES has made substantial contributions to our under- 
standing of how to improve education and is engaged in activities to address the 
challenges it faces. There is more work to be done, and as noted by our rec- 
ommendations, the Board believes some changes to ESRA would improve the func- 
tioning of lES and the Board for the continued benefit of the country. 

ENDNOTES 

1 Before the creation of lES, the Office of Educational Research and Improvement (OERI) led 
efforts to collect educational data. lES has built upon these surveys in its current activities. 

2 For example, the National Center for Education Research (NCER) serves as a hub to facili- 
tate collaboration among a diverse, interdisciplinary group of researchers who are a part of the 
Reading for Understanding Initiative (RfU). NCER is funding six research teams to advance 
theories and develop interventions to improve reading comprehension from pre-K through grade 
12. Five of the teams are testing interventions to improve reading comprehension through a va- 
riety of curricula, supplemental materials, and professional development opportunities. 

3 For instance, in September 2012, NCEE released the State and District Receipt of Recovery 
Act Funds: A Report from Charting the Progress of Education Reform — An Evaluation of the 
Recovery Act’s Role, which documents how funding was spent and includes the characteristics 
of funded schools and districts, amounts, etc. It is part of a larger study of major Federal fund- 
ing efforts and reflects an NCEE effort to get interim reports out to the public more quickly. 

’^Bettinger, Eric P., Bridget Terry Long, Philip Oreopoulos, and Lisa Sanbonmatsu. (2012) 
“The Role of Application Assistance and Information in College Decisions: Results from the H&R 
Block FAFSA Experiment,” Quarterly Journal of Economics, vol. 127, no. 3, pp. 1205-1242. 

®NCEE formed the technical methods group to work on issues and strategies that assure eval- 
uations of education interventions provide unbiased and causally valid assessments. The tech- 
nical methods working group aims to advance and provide guidance for those specialists who 
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are embarking on evaluations in education. More information is available here: http:! I 
ies. ed.gov / ncee / tech — methods / 

®For instance, in 2012, IES created the Research-Practitioner Partnerships in Education Re- 
search program, which supports partnering around issues and problems of practice identified 
by the state and local education agencies. It is administered by the National Center for Edu- 
cation Research (NCER) as part of the research grant program. 

"^One example of a Practice Guide is Teaching Elementary School Students To Be Effective 
Writers, which was released by the What Works Clearinghouse. It offers a framework and exam- 
ples, and is part of NCEE’s interest in providing practice guides that are narrowly focused and 
useful to classroom teachers. 

®For example, to explain the new NAEP Technology and Engineering Literacy (TEL) assess- 
ment, the National Center for Education Statistics (NCES), created a video. The video describes 
what the assessment covers, gives examples, and makes clear that the goal of the assessment 
is to learn whether students have the skills needed to address the challenges of our evolving 
society. Additionally, an online tutorial allows users to get a sense of the test. More information 
is available here: http:! I nces.ed.gov I nationsreportcard I tel I 

^Report from the National Center for Education Evaluation and Regional Assistance (NCEE) 
to the NBES concerning activities from March to May 2013. 

Report from the National Center for Education Evaluation and Regional Assistance (NCEE) 
to the NBES concerning activities from October 2012 to February 2013. 

The report entitled. Translating the Statistical Representation of the Effects of Education 
Interventions into More Readily Interpretable Forms, can be found here: http: II ies.ed.gov ! 
ncser / pubs 120133000 / 

i^More information is available here: http: U ies.ed.gov ! funding ! ncer — rfas ! randd.asp 
1^ Report from the National Center for Education Evaluation and Regional Assistance (NCEE) 
to the NBES concerning activities from March to May 2013. 

i^More information is available here: http: U ies.ed.gov ! ncee ! wwc ! event.aspx'?sid=28 
i®The National Center for Special Education Research (NCSER) will not hold research or re- 
search training competitions for FY 2014. Researchers interested in the study of children, youth, 
and adults with disabilities may be eligible for funding under the NCER competitions. 


Chairman Kline. Thank you very much. 

Dr. Kemple, you are recognized. 

STATEMENT OF DR. JAMES KEMPLE, EXECUTIVE DIRECTOR, 

RESEARCH ALLIANCE FOR NEW YORK CITY SCHOOLS, NEW 

YORK UNIVERSITY 

Mr. Kemple. Chairman Kline and members of the committee, 
thank you for the opportunity to appear before you today. 

Continuing the theme, from my perspective as an educator and 
researcher for more than 30 years, the Education Sciences Reform 
Act and the Institute of Education Sciences have produced incred- 
ibly important changes in the quality, quantity, and use of edu- 
cation research. 

Until ESRA and IES, education research was allowed to function 
at a standard that would never pass muster in public health, em- 
ployment and training, welfare policy, let alone agriculture and 
medicine. The dearth of good evidence in education and the inabil- 
ity to effectively communicate lessons from the little evidence that 
did exist left us with a legacy of repeatedly reinventing the wheel 
and chasing fads rather than building a reliable and useful track 
record of what worked, what did not work, for whom, and under 
what conditions. 

While I believe we are much better off than we were 12 years 
ago prior to IES, I think we are still burdened with that legacy 
from more than two generations of ineffective research. 

From a pure numbers perspective, IES has commissioned and re- 
leased findings from 90 studies that now meet the widely agreed 
upon gold standard for research — the randomized controlled trial. 
To my knowledge, that is 89 more such studies than both of lES’s 
predecessors combined had produced, and there are many more in 
the pipeline. 
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However, I believe lES’s influence extends well beyond the sci- 
entific research studies and the activities it has supported over the 
last 12 years. The principals embedded in ESRA and in lES’s work 
have also changed the way that federal, state, and local policy- 
makers evaluate and use education research. 

In New York City, where I lead a partnership with the city’s 
schools, the schools’ chancellor, most senior staff in the New York 
City Department of Education, and, yes, even the mayor now ask 
pointed questions about whether the research that they are pre- 
sented with meets the appropriate scientific standards. And I am 
also encouraged by the fact that when they do see high quality evi- 
dence they are much more inclined to use the resulting evidence 
in their decision-making even when that evidence suggests that 
programs are ineffective and should probably be discontinued. 

I think it is also worth noting that lES has supported training 
programs to develop a new generation of researchers and research 
organizations that are equipped to meet those higher standards of 
evidence. 

So in short, from my perspective the transformations that have 
occurred under lES have moved education research much closer to 
the caliber of research conducted for decades within the U.S. De- 
partments of Education and Labor and the NIH. 

Now, while applauding lES’s many important accomplishments, 
I would also describe it as a work in progress. There are several 
areas where I think the institute can be improved. 

Eirst, in some cases I believe lES has promoted scientific rigor 
at the expense of policy and practical relevance. Second, lES has 
not invested enough in building partnerships and lines of commu- 
nications between researchers and policymakers and practitioners, 
and those are just the people that should inform and benefit from 
the work that we produce. And then third, I think lES can be 
smarter in the strategies it uses to make its work accessible. 

My sense is that the original framework established by ESRA al- 
ready provides for advancement in each of these areas, and I think 
lES is headed in the right direction. The key challenges, I believe, 
lie in helping the current leadership of lES continue to make 
strides in each of the directions I just outlined. 

So I want to suggest four core principles that I believe should 
guide the further strengthening of lES, and I will call these the 
four Ps: PI, preserve scientific rigor; P2, prioritize relevance for 
policymakers and practitioners; P3, promote wider use of high 
quality evidence; and P4, prepare for the future. 

Under PI, I think it is specifically important for lES to continue 
to place a premium on funding research that establishes strong, 
causal connections between specific education policies and practices 
and the outcomes that we care most about for our students: 
achievement in literacy, mathematics, and the sciences, social de- 
velopment, and preparation for college and careers. 

P2: In prioritizing rigorous research that is more relevant to pol- 
icymakers and practitioners I think we have two challenges here. 
One is strengthening the relationships and collaboration between 
researchers, policymakers, and practitioners; and the second is en- 
suring that our studies address critical questions about how and 
why education practices and policies work or do not work, not just 
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the thumbs up did they work or not. I think in some cases lES’s 
pursuit of rigor has sometimes narrowed the scope of research to 
the thumbs up, thumbs down, did it work or not. 

And in promoting wider use and application or research, I think 
it is important that lES treat dissemination as a continuous proc- 
ess, not just an event that occurs at the end of a study, and that 
they make smarter uses of technology to promote and disseminate 
its work. 

And then finally, P4: In preparing a next generation of education 
researchers who are committed to scientific rigor, I believe lES 
should continue its support for the pre-and post-doctoral training 
programs that are ensuring that our best and brightest are going 
into the field of education research. 

Let me conclude by saying this is one of the most important 
issues — education, evidence-building — that the federal government 
can play in supporting education throughout the United States and 
I urge the committee to take up the reauthorization of ESRA. 
Thank you. 

[The statement of Mr. Kemple follows:] 

Prepared Statement of James J. Kemple Executive Director, 
Research Alliance for New York City Schools at New York University 

Mr. Chairman and Members of the Committee: My name is Jim Kemple. I serve 
as the founding Executive Director of the Research Alliance for New York City 
Schools at New York University. Our organization is a nonpartisan research center 
that conducts rigorous studies on topics that matter to the policymakers, educators 
and other stakeholders who work with New York City’s public schools. We strive 
to advance equity and excellence in education by providing evidence about policies 
and practices that promote student success. Prior to my current position, I worked 
for 18 years at MDRC, overseeing scientific evaluations of education, welfare-to- 
work, and employment and training initiatives across the country. Before that, I 
served as director for the Higher Achievement Program here in DC, and I taught 
high school math. 

Thank you for the opportunity to testify before you today. I can think of few more 
important roles for the federal government to play in education than its support for 
building and communicating rigorous evidence about what works to improve teach- 
ing and learning across the country. The current economic and fiscal environment 
makes it more important than ever to use scientific evidence to inform difficult deci- 
sions about how to allocate scarce resources, and to invest in building more and bet- 
ter evidence about the efforts we make to strengthen our schools, particularly efforts 
that flow from the federal government. 

From my perspective as an educator and researcher for more than 30 years, the 
Education Sciences Reform Act (ESRA) and the Institute of Education Sciences 
(lES) have produced incredibly important changes. Until ESRA and the creation of 
lES, education research was allowed to function at standards that would never pass 
muster with public health, employment and training, or welfare policy, let alone 
with medicine or agriculture. As a nation, we have paid a heavy price. The paucity 
of good evidence in education, and the inability to effectively communicate lessons 
from the little scientific evidence that did exist, left us with a legacy of reinventing 
the wheel and chasing fads rather than building a reliable and useful track record 
of what worked, what did not work, for whom and under what circumstances. While 
I believe we are much better off now than we were 12 years ago, we are still saddled 
by that legacy. 

Since its inception, lES has funded and released findings from 90 studies that 
meet the widely agreed-upon “gold standard” for research, the randomized con- 
trolled trial. That’s 89 more such studies than all of lES’s predecessors combined. 
However, I believe lES’s influence extends well beyond the specific research studies 
and activities is has supported. It has changed the way federal, state, and local pol- 
icymakers evaluate and use education research. 

In New York City, where I lead a research partnership with the city schools, the 
Mayor, the Schools Chancellor, and most senior staff in the Department of Edu- 
cation now ask pointed questions about whether the research that is presented to 
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them meets scientific standards. When they see high-quality research, they are 
much more inclined to use the resulting evidence in their decision-making. For ex- 
ample, New York City discontinued its use of financial incentives for school perform- 
ance in the face of solid evidence that these incentives did not improve student 
achievement. By the same token, the City has reinforced its commitment to creating 
and sustaining small schools of choice, citing scientific evidence that these schools 
are significantly improving graduation and college readiness rates, particularly 
among some of the city’s most vulnerable students. 

lES has also helped develop a new generation of researchers and research organi- 
zations that are equipped to meet those high standards of evidence. More than 25 
universities are now attracting the nation’s best and brightest to training programs 
in rigorous education science. 

While these young people come from multiple disciplines, they are committed to 
conducting high-quality education research that will be useful to policymakers and 
practitioners. 

In my view, these transformations have moved education research much closer to 
the caliber of research conducted for decades through the U.S. Departments of 
Labor and Health and Human Services and the NIH. 

While applauding these important accomplishments, I would also describe lES as 
a work in progress. There are several areas where the Institute could be improved. 
Eirst, in some cases, I believe lES has promoted scientific rigor at the expense of 
policy and practical relevance. 

Second, lES has under-invested in building partnerships and lines of communica- 
tion between researchers and the polic 3 Tnakers and educators who should inform 
and benefit from its work. Third, I think lES can do more to make its work acces- 
sible. My sense is that the original framework established by ESRA allows for ad- 
vancements in these areas. The key challenge lies in helping the current leadership 
of lES continue to make strides in each of these directions. 

I would like to organize my remarks around four core principles that I believe 
should guide the further strengthening of lES. I’ll call these the four Ps: Preserve 
scientific rigor; Prioritize relevance for policy and practice; Promote greater use of 
high quality research; and Prepare for the future. Each of these principles should 
be seen as reinforcing and complimenting the others. 

PI: Preserve the commitment to scientific rigor. Specifically, lES should continue 
to place a premium on funding research that establishes strong causal connections 
between specific education policies and practices and student outcomes we care 
about: most notably, achievement in literacy, math, and the sciences; social develop- 
ment; and college and career readiness. 

Prior to ESRA and lES, the federal investment in education research generated 
reasonably good evidence about the nature of the problems we face in our schools, 
but yielded weak and unsubstantiated claims about how various approaches may or 
may not have solved those problems and improved teaching and learning. Even 
after only 12 years of work under lES, the education research community is finding 
that many of those claims, both positive and negative, turned out to be plain wrong. 

For example, over the past 12 years the federal government allocated hundreds 
of millions of dollars for academic enhancements to after-school programs, innova- 
tive teacher professional development programs, cutting-edge adolescent literacy 
programs, and computer-based tutoring tools. Many of these investments were guid- 
ed by compelling theory but, due to the legacy of low quality research in education, 
the evidence base for their actual effectiveness was weak. Fortunately, Congress and 
the U.S. Department of Education had the foresight to make sure that these new 
investments were accompanied by rigorous evaluations under lES to learn about 
their impact on teaching and learning. Unfortunately, it turned out that most these 
initiatives, on average, had little or no impact. 

In a more encouraging example, many federal, state and local policymakers are 
currently working to expand pre-kindergarten programs. Due in part to the growing 
commitment to scientific rigor in education research, these policymakers no longer 
have to rely on one single study from the 1960s involving less than 120 toddlers 
who were exposed to an incredibly expensive set of services and supports before they 
entered school. Evidence from a growing number of credible studies is showing the 
benefits of affordable early intervention. 

Federal support for these kinds of rigorous impact studies is crucial to developing 
a more effective educational system. 

P2: Prioritize rigorous education research that is more relevant to policymakers 
and practitioners. This challenge has two parts: 1) supporting partnerships and col- 
laboration between researchers, policymakers and practitioners, and 2) ensuring 
that studies address questions about how and why education practices and policies 
work or do not work. 
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Prioritizing Partnerships: My organization in New York City, and similar groups 
in more than a dozen other cities around the country, are beginning to demonstrate 
the value of partnerships that include researchers, policymakers, administrators, 
and educators. By working together to set research priorities, interpret results and 
put findings to use, we are accelerating the pace at which research can inform policy 
and practice. 

For instance, we have worked with the New York City Department of Education 
to enhance the largest school survey in the nation, which collects vital information 
from students, parents and teachers; we have produced individual reports for 
schools involved in our studies, to help them improve in real time; and we have ex- 
amined the effects of the City’s high school choice process on low-achieving students, 
producing insights that have been useful to both the district and local community 
groups that are helping students navigate the system. This is a far cry from the 
typical end product of research, which generally targets academic colleagues and so 
often sits on our shelves collecting dust. From my perspective, lES is making strides 
toward promoting this kind of collaboration and should continue to do so. 

Similarly, the U.S. Department of Education in general, and lES in particular, 
should continue to encourage links between federal programmatic funding and lES- 
directed studies that build solid evidence about the impact of these investments. 
While lES’s independence should remain paramount, I believe a great deal of its 
struggle to be relevant can be traced to the limited role that other offices in the De- 
partment of Education (as well as the Departments of Health and Human Services 
and Labor) have had in prioritizing the research questions it pursues and to the lim- 
ited role that lES has played in getting buy-in from those offices. 

Until recently, lES has had even less interaction with State and Local Education 
Authorities, who in the end are the ultimate consumers of the evidence lES pro- 
duces. This is beginning to change, with new initiatives like lES’s Research-Practice 
Partnership program and the requirement that the Regional Education Laboratories 
(RELs) conduct their work through what are called “research alliances,” which are 
formal advisory groups comprised of state and local education polic 3 miakers and 
practitioners. I would encourage the continuation and expansion of these efforts. 

Prioritizing How and Why Questions: In my view, lES’s pursuit of scientific rigor 
has sometimes narrowed the scope of research to focus exclusively on the question 
“did it work or not?” This obscures the kind of information we desperately need to 
make education better. 

Specifically, I would encourage lES to expand its pursuit of questions about how, 
why, for whom and under what circumstances things work or do not work. These 
questions should be essential to rigorous evaluations of program effectiveness and, 
in my opinion, will be especially valuable when we find something that did not 
work. 

For example, I helped conduct a study of what are called High School Career 
Academies, a promising school reform initiative that is supposed to prevent students 
from dropping out and help them enter the workforce after high school. On average, 
we found that the programs had no effect on dropout rates (although it had large 
positive effects on workforce outcomes). However, when we dug a little deeper and 
looked at students who were at the highest risk of dropping out and were enrolled 
in the most dysfunctional schools, we found quite large dropout reductions. 

This was in spite of the fact that the programs in these schools were not very 
strong; they were just much better than anything else available to those students. 
We were able to attribute these effects to the program’s emphasis on personal rela- 
tionships and high expectations, particularly in the context of an otherwise chaotic 
environment. We would not have learned this without expanding the prevue of our 
research beyond the thumbs up or thumbs down question of “did it work or not.” 
As a result, that study is listed in the What Works Clearinghouse — lES’s definitive 
resource on scientifically validated research in education — as one of only seven with 
evidence of positive effects on keeping students in high school. 

Questions about how and why policies and practices work or do not work are im- 
portant both in the context of new initiatives and when proven practices are being 
adapted to new circumstances. In particular, lES should continue its recent invest- 
ments in what is called “continuous improvement research” — a process by which 
data collection and analysis are integrated into program development and imple- 
mentation. While still in its infancy, this seems like a promising method for using 
rigorous research to help schools become more effective over time. 

P3: Promote wider use and application of education research. Again, I believe this 
is a dual challenge: 1) treating dissemination as a continuous process rather than 
a single event at the end of a study, and 2) making smarter use of technology to 
organize and provide access to high-quality evidence. 
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Promoting Dissemination as a Continuous Process: In my experience, this also 
ties back to the importance of building relationships between researchers and the 
audiences we are trying to reach. For example, the Career Academies study I men- 
tioned earlier was a 15-year evaluation (yes, 15 years). This work, involving literally 
hundreds of contributors and collaborators, has had a profound influence on career 
and technical education. While the study found that the Career Academies produced 
sustained positive effects for on long term workforce outcomes for young people, the 
story of the study’s influence began before we collected a single piece of data. My 
colleagues and I at MDRC started this project by asking both leaders in the field 
and teachers and administrators in schools what they thought would be worth 
learning about innovative approaches to the school-to-work transition. We continued 
this dialog at each step in the study providing a wide range of audiences with early 
and long-term findings and asking for their guidance about how our work could be 
more useful. As a result, key stakeholders bought into the research process from its 
inception; they were able to confront the results, even though not all of them were 
positive; and, most importantly, they continue to this day to work diligently to re- 
form and strengthen their programs to be better aligned with what we found made 
a difference. 

Prioritizing Smarter Use of Technology: lES has led the effort to bring dissemina- 
tion of high-quality education research into the late 20th Century (although prob- 
ably not the 21st Century) through its creation of the What Works Clearinghouse — 
a compendium of studies that have been screened for scientific merit and catalogued 
by topic. It has also supported related resources like the Better Evidence Encyclo- 
pedia and the Society for Research on Educational 

Effectiveness. More recently, lES has issued a call for the establishment of a Cen- 
ter on Knowledge Utilization, whose mission will be to study how educators and pol- 
icymakers use research. I believe these are investments worth sustaining and in- 
creasing, particularly if they continue their development of research-based practice 
guides in addition to their mandate to serve as arbiters of what constitutes scientif- 
ically valid evidence. 

However, to advance the use of rigorous research, I think lES will need to make 
smarter use of technology to make this work more accessible and user-friendly. I 
do not think this is a matter of keeping better track of how many reports get pub- 
lished or how many website visits they receive. This is beyond my area of expertise, 
but I am struck by the ease with which I can find pretty useful and generally reli- 
able information about restaurant and movie reviews, and ratings of cars and appli- 
ances. I am hard pressed to believe that those of us who care about making high- 
quality research more widely available do not have something to learn from these 
efforts. 

P4: Prepare a next generation of education researchers who are committed to sci- 
entific rigor, to relevance for policy and practice, and to appl3dng what they learn 
in the field. This may be the most important legacy of ESRA and lES. 

Hundreds of talented young people have completed or are enrolled in training pro- 
grams supported by lES that place a special emphasis on teaching about scientific 
research methods. I have had the privilege of working directly with nearly a dozen 
of these young scholars, including six who are now students at NYU’s Steinhardt 
School of Culture, Education and Human Development. I have been impressed by 
their competence, certainly, but mostly by their passion for making a difference in 
schools. None of these folks will see themselves as being successful if their primary 
accomplishments are to accumulate a long list of articles in prestigious journals and 
receive tenure at a prominent university. While many of them are certainly destined 
for these accomplishments, I am convinced that they will make their biggest impact 
in producing high-quality work that education policymakers and practitioners value 
and use. 

From that perspective, I believe lES’s support for pre- and post-doctoral training 
programs and for its methodological and professional development activities are crit- 
ical investments. I particularly applaud lES’s recent call for universities to form 
partnerships with schools and school districts as key components of their doctoral 
training programs and to ensure the graduate students spend time working closely 
with administrators and teachers to learn about their needs and interests. 

In closing, I think we must recognize that despite the great leap forward precip- 
itated by ESRA and lES, the reality is that most states and school districts still 
use rigorous research in policy and administrative decisions much too infrequently. 
This is, in part, about the role that ideology and politics play (both constructively 
and disruptively), but it is also because the policy and practitioner communities not 
been very involved in the production of evidence and setting of research priorities. 
There is still very limited evidence on issues that matter to them; the evidence that 
does exist is often hard to understand and apply; and there is little incentive for 
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them to produce or use rigorous evidence. The recommendations I am offering here 
would go a long way toward addressing these issues and would help make schools 
and school districts more active partners in education research. 

Of course, there are more than 14,000 jurisdictions that make policy and admin- 
ister K-12 education in the US. The role of the federal government is limited at best 
(with only 7 percent of education expenditures covered by federal funding and lim- 
ited capacity to manage implementation). It seems imperative that a not-insignifi- 
cant portion of this limited federal investment be accompanied by two require- 
ments — similar to those we’ve seen in the Investing in Innovation Fund: 1) that 
SEAs and LEAs use federal resources to support initiatives that have credible evi- 
dence of their effectiveness, and 2) when such evidence is lacking, that they be will- 
ing to participate in rigorous research that will help fill this gap. Together with the 
four Ps I have proposed, over time, this approach could help our nation build a firm- 
er foundation of evidence and ultimately produce better outcomes for our students 
and teachers. 


Mr. Rokita [presiding]. Thank you, Doctor. 

Ms. Christie, you are recognized for 5 minutes. 

STATEMENT OF KATHY CHRISTIE, VICE PRESIDENT, KNOWL- 
EDGE/INFORMATION MANAGEMENT AND DISSEMINATION, 

EDUCATION COMMISSION OF THE STATES 

Ms. Christie. The Education Commission of the States, or ECS, 
was created by states, for states nearly 50 years ago. We help 
states learn what other states are doing, what new ideas are 
emerging, and what the research says. 

We provide unbiased information. We don’t advocate for certain 
education policies. And we don’t pick sides. 

We are the only state-focused national organization that works 
across all levels and sectors of education, from pre-K to post-sec- 
ondary, and across branches of government. 

The strength of the evidence underpinning policy levers and ini- 
tiatives is critical to the success of the policy process, but policy- 
makers seldom know what the evidence says on every issue. Much 
depends on the quality of their staff and whether they know where 
to go to find an evidence base. And yet, they have to make deci- 
sions every day, whether they can answer these questions or not. 

That is why having timely, succinct, and understandable re- 
search available is so important and why organizations like ECS 
play such a vital role. 

My role at ECS lets me sit in a national crow’s nest, watching 
the horizon for the education problems states are struggling with 
and for what they are doing to solve these problems. But it is dif- 
ficult to make sure that every education committee chair, gov- 
ernor’s advisor, state superintendent, governing board member, 
and higher education entity knows about our resources. We very 
much understand the difficulties of getting good research into the 
hands of those who can do something about it. 

The Institute for Education Sciences has some entities available 
to attempt to address some of these needs, including regional edu- 
cation laboratories, commonly known as RELs, comprehensive cen- 
ters, and content centers. In the past the production from RELs 
and centers seemed uneven. Resources seemed to take a long time 
to come to fruition, and by the time they did sometimes the window 
of opportunity to inform decisions had passed, and decision-makers 
were not always at the table to set the agenda. 
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Today the What Works Clearinghouse is building a strong base 
of easily accessible program reviews and pointing toward interven- 
tions that work and that don’t work. Summaries are now less aca- 
demic and easier to follow. Practice guides provide good direction. 
Conclusions are presented in a more straightforward manner. 
Readers can actually easily access areas by topic. 

Conclusions, I am sorry. lES might consider how to more clearly 
distinguish between findings regarding whether studies meet 
standards of evidence and whether programs actually impact learn- 
ing. Overall, however, the site has improved greatly. We link to the 
clearinghouse by issue area, which is an efficient way to imme- 
diately capture updates for our constituents. 

In addition, content centers on issues of importance to states, 
such as turnarounds and state capacity, can be spot on for meeting 
state needs. But lES could work to ensure that vetting and review 
for activities and outputs does not inhibit the development of time- 
ly, relevant, digestible research and assistance. 

The Best Evidence Encyclopedia, which has funding ties to lES, 
is another excellent resource. And my understanding of the new 
breed of RELs is that they are working to establish research alli- 
ances across those states with state leaders at the table. 

To put these entities in perspective, though, I would like to high- 
light what is good about one of my favorite sources, the National 
Bureau for Economic Research. Nearly every week I get a message 
highlighting several of their new studies. 

These studies are relevant to the problems I see states struggling 
with, such as compulsory attendance. They are timely; they are 
prolific. lES could look to this model for improving relevance and 
timeliness. 

Like most academic studies, these are so academic that people 
are not going to read them, and for the most part they are not 
openly accessible. So ECS is working to translate studies like this 
and capture the key findings and recommendations in our research 
studies database. We organize them by frequently asked questions 
such as, “Preschool: How prepared do teachers need to be?” 

Since 2008 we have entered 193. We are very thankful for the 
GE Foundation for supporting this work. 

The reason for this effort is clear: When busy people ask, “What 
does the research say?” any response to that question needs to be 
timely, relevant, digestible, and trusted. 

Here are the four final points I would like to make. 

One: Research matters not only to those implementers in the 
field — the superintendents, principals, and teachers — but to those 
who are committed to improving the system of education. 

Two: The gold standard matters, but while optimal, it is not al- 
ways possible. The real world will continue to demand that policies 
be crafted based on hypotheses that are relatively well supported 
by evidence or whether early evidence is simply promising. lES 
could do a better job of ensuring that topics fit with what matters 
to states and that its research helps answer not only which pro- 
grams work but also which policies or state investments hold prom- 
ise and which elements of those policies matter most so that state- 
level elected officials might act on them. 
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Three: lES needs state leaders to perceive it as an unbiased, hon- 
est broker, so increasing the independence of lES could be key. 

And four: lES should consider a coordinated effort to trans- 
parently evaluate and hold itself accountable on a set of perform- 
ance measures that are important to states. 

Thank you for letting me be here. 

[The statement of Ms. Christie follows:] 

Prepared Statement of Kathy Christie, Viee President, Knowledge/Informa- 
tion Management & Dissemination, Education Commission of the States 

Me. Chairman and Members of the Committee: My name is Kathy Christie. 
I am Vice President, Knowledge/Information Management & Dissemination for the 
Education Commission of the States (ECS). The Education Commission of the States 
(ECS) was created by states, for states, almost 50 years ago. 

Since 1965, ECS has worked with state policymakers to improve America’s public 
P-20 education system. We provide unbiased knowledgeable advice to state leaders 
and help them to learn from one another’s experiences. What makes ECS unique 
in the crowded education policy arena is that we work with policymakers, research- 
ers, thought leaders and practitioners across all levels and sectors of education — 
from pre-K to postsecondary — and across branches of government. We are the only 
state-focused national organization that brings together governors, state legislators, 
chief state school officers, higher education officials, and business leaders to advance 
policies that improve our educational system. To accomplish this work, we under- 
take the following kinds of activities: 

• We help states learn what other states are doing, identify best practices, what 
new ideas in education are emerging, and what the research says. 

• We provide unbiased information. We don’t advocate for certain education poli- 
cies and we don’t pick sides. 

• Our website is one of the best in the country to find information on hundreds 
of education issues. You can check out our website or call us directly; either way, 
we will provide the information that’s needed. 

Most people are not aware that it was ECS — the only nationwide interstate com- 
pact for education — which was responsible for the creation of NAEP, the National 
Assessment of Educational Progress. It is this kind of state collaboration and state 
insight that makes ECS unique. The message I bring here today is about the power 
of state-level leaders to make a difference in the educational opportunities available 
to children. 

State Leaders Influence System-Level Reforms 

Policymakers at every level in states — chief state school officers, governing board 
members, legislators, and governors — all have a role to play in developing and im- 
plementing education initiatives. Very often these various policy actors have dif- 
ferent opinions about how laws should be shaped. Consequently, evidence related 
to the hypotheses that underpin state-level policies and evidence on the relative ef- 
fectiveness of state-level initiatives are critical to the success of the policy process. 

They hope, and we do too, that the decisions they make will drive improvements 
in what can be a large, bureaucratic system. A constituent of ours once described 
the education system as a big pillow. You punch it — and make a big dent, but 
gradually that dent disappears and the pillow puffs out to its original shape. You 
punch it again, making another dent. But after a bit the pillow again puffs back 
out to its original shape. We all understand how tough it is to make real change. 

State-level policymakers seldom know what the evidence says on every issue. 
Much depends on the quality of their staff and whether they know where to go to 
find an evidence base. Even when they have capable staff with sufficient time, it 
can be a challenge to gather and prepare evidence in a way that allows policy- 
makers to quickly understand whether it is sufficiently robust to support the deci- 
sion they will make. 

When considering a hypothesis behind a bill, legislators routinely ask: What is the 
level of evidence supporting this proposal? Is it minimal, with a higher risk to suc- 
cess? Is there some evidence available, but it fails to highlight the most critical fac- 
tors that need to be in the policy? Or is there extensive preliminary research and 
piloting, with interventions that have been aligned at all levels and across agen- 
cies — a sufficiently robust knowledge base on which to guide large-scale decisions? 

And yet state legislators have to make decisions every day — whether they can an- 
swer these questions or not. That’s why having timely, succinct and understandable 
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research available is so important and why organizations like ECS play a vital role 
in state-level education policy. 

National Perspective 

My role at the Education Commission of the States lets me sit in a national crows’ 
nest, watching the horizon — across state boundaries — for the education problems 
states are struggling with and for what they are doing to solve those problems. ECS 
scans news clippings every day and pushes the most relevant out via email — every 
day. We track the policies that state legislators are enacting, and we add them 
every week to the most extensive, freely-available database of its kind in the coun- 
try. Every day we are culling from the professional and academic literature and 
pushing the best back out via our web site. But it is difficult to make sure that 
every chair of an education committee, every governor’s education policy advisor, 
every state superintendent, every state board member and every higher education 
agency head knows about those resources. We very much understand the difficulties 
of getting good research into the hands of those who can do something about it. And 
these challenges don’t even begin to touch the difficulty of reaching every legislator 
and agency head and governing board member across the country. 

What I have learned in over 20 years with ECS is that we reinvent the wheel 
time and time again. Policymakers don’t pay enough attention to history. We might 
read the research and go, “oh, yeah, that’s right” but then we too often jump at 
some new shiny, glittery answer or lobby for a new research study rather than tak- 
ing time to unearth the root cause of a problem or step back to analyze the existing 
research — the research that while there, might not be broadly available or is so in- 
comprehensible, we don’t know what to make of it. 

The Institute of Education Sciences has some entities available to attempt to ad- 
dress some of these needs, including Regional Education Laboratories (commonly 
known as RELs), comprehensive centers, and content centers. Past RELS seemed 
uneven in production of resources, particularly those that might remain relevant 
and useful long after individual instances of technical assistance or convenings. Re- 
source development or projects seemed to take a long time to come to fruition, and 
by the time they did, sometimes the window of opportunity to inform decisions had 
passed. And decision-makers were not always at the table to set the agenda. 

Today the What Works Clearinghouse (WWC) is building a strong base of acces- 
sible program reviews and pointing toward interventions that work — and that don’t 
work. WWC has improved over time. Summaries are now less academic and easier 
to follow. Practice guides provide good direction for practitioners. Conclusions are 
presented in a more straightforward manner. Readers can easily access areas re- 
lated to specific topics. lES might consider how to more clearly distinguish between 
1) findings regarding whether studies meet standards of evidence and 2) evaluations 
of actual program effects on learning. Overall, however, the site has improved great- 
ly. ECS is able to link to WWC issue areas via relevant topic areas (e.g., literacy) 
on www.ecs.org, so as studies are added to WWC, it is not necessary for us to add 
each new review to our site. This is efficient and immediately captures updates for 
our constituents. 

Content centers that focus on topics that matter to states — turnarounds and state 
capacity, for example — can be spot on for meeting state needs. lES might review 
processes to ensure that vetting and review processes for activities and outputs of 
these new centers and for the new RELs does not inhibit the development of timely, 
relevant, digestible research and assistance. 

The Best Evidence Encyclopedia is another excellent resource: a free web site cre- 
ated by the Johns Hopkins University School of Education’s Center for Data-Driven 
Reform in Education (CDDRE) under funding from the Institute of Education 
Sciences. And my understanding of the new breed of Regional Education Labs 
(RELS) is that they are working to determine what states need and helping to es- 
tablish research alliances across those states — with state leaders at the table. But 
like many education policy organizations, including ECS, RELs may struggle to 
raise awareness of available resources and how they can support their states and 
school districts. 

From my crows’ nest, I seek out and cull research reports from the comprehensive 
centers and the content centers — although some are more prolific than others. 

To put this comment in perspective, I’d like to highlight what’s good about one 
of my favorite entities — the National Bureau for Economic Research. Every week I 
get an email summarizing the education studies they have completed. Every week. 
Not four per year. Not one per month. Every week. They’re called Working Papers, 
so one’s immediate assumption is that they are food for thought — not to be consid- 
ered definitive, but findings to think about. Why are they so compelling? They are 
relevant to the problems I see states struggling with — like compulsory attendance, 
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for instance. Does the age at which kids start school matter? Does the cut-off age 
(5 by September 1, for example) for attendance matter? Does the upper compulsory 
attendance age make a difference over the course of a lifetime? NI5ER studies are 
relevant and timely. They look at the types of issues that governors and state legis- 
latures can influence via policy. They are prolific producers of what most would 
agree is quality research. 

These studies are dense. Their titles are often just abysmal and if they were mov- 
ies, no one would buy tickets (e.g. Interpreting the Evidence on Life Cycle Skill For- 
mation), even though they have much to offer. They are so academic that most state 
policymakers — or their excellent staff — are not going to read them. The same goes 
for studies coming out of AERA. And for the most part, the studies are not openly 
accessible. But they have something to say to you. And to state legislators. And to 
governors. So ECS is working to translate studies like this and capture the key find- 
ings, recommendations, and implications for policy in its Research Studies Data- 
base. We organize them by frequently asked questions such as “Preschool: How pre- 
pared do teachers need to be?” or “High school curriculum: How important is rigor?” 
Since 2008, we have entered key findings and policy implications from 193 studies 
into The ECS Research Studies Database. We are very thankful to the GE Founda- 
tion for supporting this work. 

The database could be easier to use. It could be “prettier.” We have created stand- 
ards for inclusion, and an important standard is that studies need to have appeared 
in peer-reviewed journals. But we do make exceptions. 

For example, I personally mined Crossing the Finish Line — a 2009 book by Wil- 
liam Bowen, Matthew Chingos and Michael McPherson — names probably familiar to 
many in this room — that looked at what impacts student persistence and success 
across postsecondary institutions. Compelling, compelling statements built from sta- 
tistical analyses are buried throughout, but they were most certainly read by more 
academics and higher education leaders than by policymakers. The authors’ analysis 
supports, for example, the clearly articulated statement that “both parental edu- 
cation and family income are strongly associated with graduation rates even after 
controlling for related differences in student characteristics, particularly academic 
preparation. However, family income, not parental education, is primarily respon- 
sible for the overall relationship between SES and time-to-degree.” 

If another credible, vetted resource emerges to counter such findings, ECS will not 
hesitate to include it in the database as well. 

As exceptions to the peer-review rule, we include NBER studies, many of which 
eventually are published in peer-reviewed journals — but in the interim, they reflect 
the food for thought that state leaders need. 

Do enough state leaders know about this resource? No. Have we been successful 
in marketing it? Probably not. We always include new studies in our weekly e-news- 
letter, e-Connection, and they always get the most hits; regrettably, we have not 
been particularly successful in marketing its availability. 

The reason we acted, though, is clear. When a consistent element of questions is 
“what does the research say?” the response needs to be timely, relevant, digestible, 
and trusted. 

Here are the four final points I would like to make: 

1. Research matters not only to those implementers in the field — the superintend- 
ents, principals and teachers — but to those who are committed to improving the sys- 
tem of education. 

2. The gold standard matters. But while optimal, it is not always possible. The 
real world will continue to demand that policies be crafted based on hypotheses that 
are “relatively well” supported by evidence or where the early evidence is “prom- 
ising.” lES could do a better job of ensuring 1) that topics fit with what matters 
in states; 2) that its research helps answer not only “which programs work” but also 
which policies or state investments hold promise — and which elements of those poli- 
cies matter most so that elected officials might act on them. 

3. lES needs state leaders to perceive it as an unbiased, honest broker, so increas- 
ing the independence of lES could be key. 

4. In that regard, lES might want to consider a coordinated effort to transparently 
evaluate and hold itself accountable on a set of performance measures that are im- 
portant to states. 


Mr. Rokita. Thank you, Ms. Christie. 

Thank the witnesses for their testimony. 

I now would like to recognize Chairman Walden, Walberg for 5 
minutes. 
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Mr. Walberg. Thank you, Mr. Chairman. Put a W in front of it 
and we are always at the end, so we all look the same. Thank you. 

Appreciate the panel being here today, and in the process of 
doing a lot of reauthorization of laws that are in place, agencies 
that are in place, the question continues to come to my mind of 
why. 

Ms. Christie, should reauthorization of Education Science Reform 
Act take place? 

Ms. Christie. I think lES is an incredible resource. Like all of 
us, it can get a lot better. 

Mr. Walberg. Mr. Scott, I would ask you the same question. 
From your perspective with GAO, should reauthorization of the 
Education Science Reform Act — of course that is very pertinent to 
lES — take place? 

Mr. Scott. Well, I think as our work demonstrates, lES has 
played a significant role in improving the quality and the rigor of 
education research, and given the number of efforts that are cur- 
rently underway to reform the U.S.’s educational system, it is criti- 
cally important that we continue to have some vehicle such as lES 
to ensure that the various programs that are underway can be 
properly evaluated. 

Mr. Walberg. Ms. Christie, is it the responsibility of the federal 
government to do this research, or could it be done equally as 
well — maybe more efficiently — at the private sector level and, in 
fact, at the states in competition for educational quality, promoting 
research entities that would definitely be looking at states and 
their responsibility for education? 

Ms. Christie. We love to highlight states that in every bill put 
in an evaluation component to — so that they are modeling contin- 
uous improvement to their constituents, their schools, their dis- 
tricts. Many times it is the cost component of that that is difficult, 
and there are other times when if there are common problems 
across the states — and there are; they are nearly all common even 
though we all would like to think we are absolutely unique — but 
when you have common problems that is the power of ECS. You 
do things as a collaborative. You do things that keep the scale 
within reach. 

And I think the federal level has always had a role in keeping 
the spotlight on equity and ensuring all kids have opportunities, 
and I think that there are certain things that are probably fitting 
for both to do. 

Mr. Walberg. I guess my concern comes back that also on the 
federal level bureaucracies develop that take $600 million of re- 
sources that certainly could be effective at a private sector entity, 
looking for best practices with those unique students we have. And 
yet, as you I think accurately state, with their uniqueness there is 
a great deal of sameness as well, that parents, local school boards, 
school districts, superintendents, teachers want to achieve in the 
outcome of students. 

And I guess I am not hearing, as of yet, a strength in the answer 
that this could not be done in somewhat a market-based approach 
at the local level — of states specifically — of doing the research that 
is necessary with best practices that are out there and a clear un- 
derstanding that we need to achieve those and how do we do that. 
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So respond to that, Ms. Christie, if you would, please? 

Ms. Christie. The big component here is trust. So if you have 
a lot of private sector folks doing research, which even right now, 
I mean, when lES contracts with REL providers, those are basi- 
cally private contractors. 

There needs to be a sense that what is coming out is unbiased 
and that it can be trusted. And I worry that if that gets outside 
of an independently verified group that you could lose that trust, 
and I think that would be a huge gap, then, in the research. 

Mr. Walberg. Well, Mr. Chairman, I would yield back my time 
after saying that if we are talking about trust in context with the 
federal government and independence in context with the federal 
government we have a major hurdle to get over. 

I yield back. 

Mr. Rokita. I thank the gentleman. 

Ranking Member Miller is recognized for 5 minutes. 

Mr. Miller. Thank you very much. 

Sounds like there is a considerable amount of trust here at this 
table in this process, with some suggestions for improvement — but 
not eradicating the concept that lES has been put here. 

Dr. Kemple, you — I think the panel seems to agree that we are 
doing pretty well on the rigor side here. The questions is about how 
do you get that dissemination to take place that is actually useful 
at the state level or even at the district school level. 

And that has always been a problem for me. I would say before 
we got into this process, what we had was a landscape littered with 
5-year pilot projects that just died and nobody ever asked what 
happened to them or did it work or not, and then somebody came 
along over and over and over again, very often with the same pilot 
project just in a different state, different district around here. 

So how are we doing on this dissemination in terms of following 
through on it — not just getting it there, but then what is done to 
help people if the state decides they want to go through it? I mean, 
we are sort of going through this with Common Core. Our governor 
just put up $1 billion to help the districts with the implementation. 
I mean, this is a big hurdle in terms of taking new findings of rig- 
orous research and suggesting to people, “This may be a better way 
to do something than you have been doing for 15 years the other 
way.” 

How are we doing on that hurdle? 

Mr. Kemple. I think this is a two-fold problem. I think it starts 
with thinking about dissemination and promoting the use of re- 
search as a process rather than as an event. Too often we place 
much too much emphasis on waiting until the study is done until 
we actually think about what we have been able learn from that. 

I think. A, it is important to think about dissemination as it is 
starting before even one iota of data is collected to make sure we 
are asking the right questions; I think it is important that we in- 
corporate the policymakers and the expertise and the practitioners 
who are part of the formation of the research projects or the evi- 
dence-building process in a continuous way throughout a project so 
that we are feeding the information that we are learning in process 
into a decision-making process and into a learning process. 
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And I think thirdly, it is important to make sure that as we 
package results, that as we put results together, we are translating 
those into practice guides so that we are being led by the best evi- 
dence possible in a way that is actionable and that makes the prac- 
titioner community — or puts the practitioner community in a posi- 
tion to learn from both the work in progress and the work at its 
conclusion. 

Mr. Miller. Dr. Long, this is your field. How is it going? 

Ms. Long. I would say that lES is doing a wonderful job in terms 
of production and leveraging the field to produce the research. I 
think the dissemination part of it is a work in progress. 

And actually, as the board and lES have talked about it much 
more in terms of communication, which is a two-way street be- 
tween lES, the work that is produced, and what is known with the 
field and what the field needs in order to do better work. 

Part of what is involved with that is translation — taking very rig- 
orous research, which education has gone forward tremendously in 
having causal results, but translating that for the layperson, for 
the teacher who is in the classroom so they know how to use it, 
and then learning how to disseminate it. 

lES needs to do more 

Mr. Miller. Ms. Christie, is that your job? At that stage is that 
your job, the handoff here to the states? 

Ms. Christie. We hand off everything we can get our hands on 
to the states, yes 

Mr. Miller. No, but I am just asking, you know, you get this 
new research, you get a model for dissemination or to the impor- 
tance of this research, and I just want to — so where do the states 
pick this up and make a decision? 

Ms. Christie. Yes, we 

Mr. Miller. Not all research is welcome, you know, I find from 
time to time in the education establishment because it suggests 
substantial change — significant change. So how do the governors 
pave the way when you have rigorous research that suggests you 
have got to change directions? I mean, we are now, what, 2 years 
into the Common Core with governors modifying and rethinking 
the model back and forth to where we are today. 

Ms. Christie. Well, if I could make one suggestion, so much of 
what is in the What Works Clearinghouse, for example, is about 
the programs, not necessarily about those big policy issues that a 
governor needs to take up. 

But I think research is welcome, it just needs to be crafted in a 
way that folks like me — I mean, I am just a translator; I am not 
an academic — that I can understand and then I can put into prac- 
tice. 

Mr. Miller. Well, Mr. Kemple, how do we get the feedback loop 
back to the researchers and the disseminators at lES based upon 
what you have learned in modification? Because if you look at 
teacher sites, teachers are always modifying somebody else’s lesson 
plans, somebody else’s approach, and it is rather an interesting 
process. How do you get what you are doing in New York back to 
the lES and others about how this is going? 

Mr. Kemple. Well, I think just in the way that a teacher collabo- 
rative, as you mentioned, works, as someone is learning from — tak- 
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ing what one teacher has done and they are building from another, 
those folks are all at the table together talking both about what 
their goals and objectives are, what kinds of problems they are fac- 
ing in the classroom, and what kinds of different solutions they 
have each come up with on their own. Again, I think this is a proc- 
ess of making sure that all of those folks are somehow at the table 
as we discuss the nature of the problems that we face, the solutions 
we have each come up with, and then cycling that back into an ef- 
fort at continuous improvement. 

Mr. Miller. Thank you. 

Mr. Rokita. I thank the ranking member. 

Mr. Tierney is recognized for 5 minutes. 

Mr. Tierney. Thank you. 

Thank the witnesses for their testimony. This, I think, follows 
along with Mr. Miller’s questions a little bit, and it may — I think 
it is a fair question to ask. Can any of you tell me about a program 
or practice that was actually affected by the research — one that 
was either scaled up or eliminated or changed in some way? 

Mr. Kemple. Sure. I was privileged to lead a study of what are 
called high school career academies, which are high school reform 
initiatives aimed at dropout prevention and helping young people 
make healthy transitions into the workforce and to college. This is 
a 15-year study — yes, 15 years. So if we had waited 15 years for 
this to translate into some change in practice I think we would 
probably have lost any momentum or chance of making a dif- 
ference. 

But this was an initiative that found little or no impact on col- 
lege readiness, although many of the young people in the study 
went on to college, but found massive effects on their capacity to 
find good jobs, keep good jobs, and climb a career ladder in the 8 
years following high school graduation. We were able to form a coa- 
lition of stakeholders in school districts, at the state level, and 
among expert organizations across the country that were part of 
helping us form that research project and part of the process that 
we used to disseminate what we were learning over the course of 
the 15-year study, so that by the time we got toward the end of 
that project many of those groups had already begun to synthesize 
the work into the creation of organizations that were aimed at sup- 
porting the standards for implementing — for creating and sus- 
taining these career academy programs and for a continuous im- 
provement process to work on some of the weaker aspects, such as 
the academic curriculum and the college access programs. 

Mr. Tierney. So 15 years — that is before the 2002 

Mr. Kemple. That is right. 

Mr. Tierney [continuing]. Authorization of this act. So was that 
done under this act or something else? 

Mr. Kemple. That was actually done independently through pri- 
vate funding. 

Mr. Tierney. Okay. Well, my question to you is can you give me 
an example of something that occurred under this act that either 
changed in some way a program or a practice 

Ms. Christie. Dr. Long? 

Ms. Long. Yes. I can speak about some of my own work that was 
supported by lES. 
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Mr. Tierney. Please. 

Ms. Long. Working with two of my colleagues, Eric Bettinger 
and Phil Oreopoulos, we worked with tax preparers to help families 
fill out their student financial aid forms, the FAFSA. And so after 
completing their taxes we only needed 5 additional minutes to pre- 
populate the FAFSA using software and then asking them a few 
additional questions as well as giving them information about col- 
lege prices and what they needed to do in order to access those in- 
stitutions. We found huge results in this randomized controlled de- 
sign of getting many more students into college by just offering 5 
minutes of assistance. 

This report was first released in 2009, right around the debates 
about FAFSA simplification, was it worthwhile, and we have subse- 
quently gotten an additional grant from lES to continue to see how 
we can expand these kinds of services and community tax sites 
around the country. 

Mr. Tierney. And did the FAFSA reforms in any way play off 
of those findings, and did they 

Ms. Long. We certainly fed that information into the debates 
and received many calls from both states as well as many members 
of Congress around the federal government to understand that, yes, 
information is a huge barrier. This was a randomized controlled 
trial, simplifying things and giving assistance, and it turned out it 
was very cost effective. 

Mr. Tierney. Dr. Scott, or Mr. Scott, do you find enough of those 
examples in your work, in your review of this agency to continue 
to warrant its continuation? 

Mr. Scott. I think, as we noted in my statement, lES is uniquely 
positioned to act in this area. I think it is important, though, that 
as we also pointed out, that they take some steps to improve. For 
example, in the areas of continuing to get feedback from practi- 
tioners and policymakers, continuing to hold various aspects of 
their operations accountable, such as the RELs, and then con- 
tinuing to measure and report out on their activities. So I think 
lES is uniquely positioned to contribute in this area but there also 
needs to be some improvements to their operations and account- 
ability. 

Mr. Tierney. Thank you. 

Do practitioners — do actual teachers and superintendents and 
principals — have a way of connecting with lES and looking at these 
reviews periodically and interacting? Is there some setup to encour- 
age that? 

Ms. Long. There are several efforts. There is information that is 
available online. There are constant efforts to translate and make 
this information available. For example, the practice guides have 
been cited, which really break down the research to what teachers 
can do in their classrooms. 

But this is a work in progress. Much more needs to happen, and 
lES is taking steps to get more feedback from the field about what 
they need and how they need to use that information. 

Mr. Tierney. And are you working with colleges that teach peo- 
ple to become teachers as to how they might access a tool like this 
and make it part of their overall practice as they go out and teach? 
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Ms. Long. That is one of the audiences, hut much more could he 
done with them. 

Mr. Tierney. Thank you. 

Thank you, Mr. Chairman. 

Mr. Rokita. I thank the gentleman. 

Ms. Bonamici is recognized for 5 minutes. 

Ms. Bonamici. Thank you very much, Mr. Chairman. 

And thank you to the witnesses for being here today for this dis- 
cussion. I, especially as a member of this committee, truly believe 
that education is an investment in our country’s future and so it 
is important for us to get the policy right here. 

And, Ms. Christie, I want to say that as a former state legislator, 
thank you. Frequently relied on the Education Commission of the 
States and your work there. 

I especially appreciate your pointing out in your written testi- 
mony that you have learned over the years that we tend to re- 
invent the wheel, and also pointing out that it is important to take 
the time to really unearth the root cause of a problem, and I hope 
that this hearing today will help with both of those areas — help us 
not reinvent the wheel, but also not neglect the importance of look- 
ing at the root cause of the problem. And I have often spoken about 
the importance of, for example, looking at poverty and homeless- 
ness and how that affects students in school. 

So I wanted to talk a little bit about what many of you have 
mentioned both in your written and oral testimony, and that is the 
relevance of research. So I am from Oregon, and we have the Or- 
egon Leadership Network Research Alliance, which is under our 
Education Northwest, and they have really been working to bring 
together the practitioners on the ground with the policymakers, un- 
derstanding that the research needs to be accessible to both of 
these groups, and I agree. 

So I would like to talk a little bit about what relevance means. 

And, Dr. Kemple, you said that many states and school districts 
don’t use rigorous evidence because they have not been involved in 
producing or guiding what research is done. So you also mentioned 
that it is important to prioritize educational research that is rel- 
evant to policymakers and practitioners. 

So can you all talk a little bit about what does relevant mean 
and who is determining relevancy? So are the practitioners and the 
policymakers all saying that the same things are relevant? And 
how do their views differ? 

And then who ultimately makes that decision? If we are saying 
we want relevant research, who is going to make that decision? Is 
that going to be us in — as policymakers, is it going to be the practi- 
tioners, or is it going to be the researchers, and why? 

Mr. Kemple. So briefly, I think relevance means two things. One, 
I think it means, whose questions are we asking as researchers or 
how do we — whose questions are we prioritizing as researchers? I 
think that is a two-way conversation. I think it has to include peo- 
ple who are the ultimate consumers of their research. If it is a 
question for Congress about whether they would like to make sure 
that an investment in early childhood education, adolescent lit- 
eracy, afterschool programming is paying off in terms of improv- 
ing — achieving the goals that it was set out to, either improving 
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teaching or improving learning among young people, we ought to 
know that those are priorities for what gets learned and that con- 
versation needs to happen between the people who are making the 
tough decisions about how to allocate scarce resources for the ben- 
efit of children. 

Secondly, though, I think it means that we have to place a high- 
er priority on the questions about why and how and under what 
conditions are investments in education improvement make a dif- 
ference or don’t make a difference. In my view, it is just as impor- 
tant to find out something doesn’t work as it is to find out that it 
doesn’t work — or that it does work, and — but going beyond the 
thumbs up, thumbs down to say, “Why did this work? What were 
the causal mechanisms? What was the appropriate context?” Those 
are the things that will matter most and translate well enough in 
Oregon as well as New York State or Arizona or Texas. 

Ms. Bonamici. Thank you. 

And from the other witnesses, do you want to give me some 
input on how those determinations are made about what research 
is done? How is it determined that it is relevant? And how can we 
better improve those — that communication between the practi- 
tioners and the policymakers and the researchers? 

Ms. Long. I think the importance of partnerships cannot be un- 
derscored enough. And this is a movement that lES is going to- 
wards, that there has to be a feedback mechanism where you are 
doing research in partnership with the field, with the teachers, the 
schools, the universities, so that as you learn as you go along and 
it is a continual improvement type of model. Because what is rel- 
evant can also change very quickly. 

At the same time, you need a federal organization that is going 
to be objective and look at the national good and say, “This is what 
is important for the country. This is the information that we all 
need to know.” And some of the myths that we have had before, 
we need to realize those don’t work and other things do, while also 
feeding that information to people who are making decisions — the 
policymakers — to understand, given our limited resources, we have 
to make decisions, we have to prioritize. So that is also very impor- 
tant and relevant. 

Ms. Bonamici. Thank you. 

Ms. Christie? 

Ms. Christie. Sure. I would like to push on a little bit different 
way that might not have been talked about in the past. 

There are groups like ECS and the National Conference of State 
Legislatures, National Governors Association, Council of Chief 
State School Officers, the state boards organization, that if some- 
one reached out to all of us we could weigh in on what we hear 
from the states. I mean, I watch every day e-clips across the coun- 
try, we have a policy database that even someone at ECS could put 
together a team and sit down and actually look at what are the 
policies that are being enacted and what are states struggling 
with? Because they are like the tip of the iceberg for what people 
need to know. 

Ms. Bonamici. Thank you. 

I see my time is expired. Thank you, Mr. Chairman. 

Mr. Rokita. Thank the gentlelady. 
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Ms. Foxx is recognized for 5 minutes. 

Ms. Foxx. Thank you, Mr. Chairman. 

And I want to thank our panel for being here today. 

And I want to say, I want to associate myself with the train of 
questioning that Mr. Miller was going at before, I believe, and 
other folks, in terms of the way we are looking at this issue, and 
particularly in the way of application. That old saying about deja 
vu all over again, I have been in this business for a long time and 
this panel — we could be in a time warp here, actually, because I 
believe I first heard this kind of debate and this kind of discussion 
back in the 1960s and maybe 1970s and 1980s when I was on the 
school board, when I did my master’s degree. 

We have been dealing with this issue forever in terms of how do 
we get the research — how do we get the knowledge that we have 
applied appropriately in the places where it can do the most good? 
And it seems as though we haven’t figured that out yet. 

I mean, people have been decrying the fact that folks in edu- 
cation just ignore the research and the results. 

So what I would like to ask each one of you is do you have some 
examples of where you have seen the research that we — of pro- 
grams that work well, methods that work well — where has it been 
applied well, appropriately, and how do we replicate those situa- 
tions without it costing a lot of money? 

Anybody? 

Mr. Kemple. I guess one example I would point to for now is — 
and highlight the fact that I think one thing that really does dif- 
ferentiate us now from the 1960s, 1970s, 1980s, and even 1990s is 
that what was not on the table then was whether we had evidence 
we could really believe. I think by and large we were making big 
investments in research that really didn’t provide us with any 
causal link between the programs and policies that were being paid 
for and the outcomes that we cared about for the young people in 
the schools. 

I think one — what we are starting to see — and among those 
things were teacher preparation. I think what we see is a long lit- 
any of research that suggests that it doesn’t matter how much — 
where people were trained, what their test scores were coming out 
of graduate schools of education, and whether, in fact, they had a 
master’s degree. What mattered most was their experience in the 
classroom and the way that they were mastering their subject mat- 
ter material. 

We now have some pretty solid evidence that some alternative 
routes to certification or entry into teaching are turning out to be 
as if not more effective than traditional routes through schools of 
education. I think we are starting to see a proliferation of strate- 
gies to draw in talented people from across the country to opt for 
teaching as opposed to other highly prestigious occupations, and 
some of that is growing out of the research that suggests that ini- 
tiatives like Teach for America or the New Teacher Project are pro- 
ducing positive gains for young people in classrooms. 

Ms. Long. To build from that, even today there is the release of 
a report looking at alternative certification programs that is being 
released that was partly funded by lES, and it has completely 
changed the way people discuss what is a high quality teacher. 
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Many of the old models that we have heen using for years of just 
get a master’s degree have heen called into question and because 
of the data that lES collects we know that the way that teachers 
are distributed across schools varies incredibly. Where there are 
high-need schools with students that are suffering, we have to fig- 
ure out policies of how to get teachers there. 

I think the other thing lES has definitely changed is the way 
that people are trained to do education research and who is at- 
tracted to do education research. You now see in graduate schools 
of education people who used to be former teachers, who worked in 
community-based organizations, who now, combined with all of 
these rigorous tools, are doing research in a very different way 
than they did in the past. 

The other thing that is very different now is technology, and this 
is something that lES is grappling with: How can you use tech- 
nology to communicate and get feedback from the field? And that 
is everything from building a Web site, which is certainly not 
enough, to podcasts and other kinds of uses of technology, and we 
have seen several examples — everything from how they are commu- 
nicating with teachers to parents to trying to distill some of this 
information so that they can use it. 

Ms. Christie. Can I weigh in? 

The early literacy and the early learning research I think has 
had a dramatic effect. The problem is I graduated with a teaching 
degree in 1970, and at that time I knew that the first three years 
of a child’s life were the most important ones, and that is why later 
in my career I decided to stay home with my kids for 12 years. 

So you are absolutely right about the recycle. And ECS had a 
huge early learning initiative in the 1970s, so we have known. The 
trick is, exactly, why do we keep moving on to the next easier thing 
to do than getting back to what we need to do, how do we push 
that back out, how do we make sure we are actually making 
progress? And those are the really tough questions, I believe. 

Mr. Rokita. I thank the gentlelady. 

Ms. Eoxx. Can I indulge for 10 seconds? 

Anybody who comes in this room who is in the education busi- 
ness and uses the word training will get my lecture. We train ani- 
mals and we educate people. So those of you who are in the busi- 
ness of education, I would ask you to not talk about training but 
talk about education. 

Mr. Rokita. I thank the gentlelady. 

Mr. Scott is recognized for 5 minutes. 

Mr. Scott of Virginia. Thank you, Mr. Chairman. 

And I have some of the same questions that the ranking member 
had about what you actually do with the information. Eor example, 
I notice that the What Works Clearinghouse studied many dropout 
prevention programs. Exactly what do we know about dropping out 
and what can be done about it on a local level? 

Ms. Christie. Can I jump in? 

Mr. Scott of Virginia. Sure. 

Ms. Christie. I think the data that is some of what lES is fund- 
ing — for example, with early warning indicators and that sort of 
thing — is a tremendous leap forward, and I think states are paying 
a great deal of attention to that, trying to figure out how to even 



38 


get those early warning indicators down into the very early grades 
to identify kids very early on. 

The interventions that are in the What Works Clearinghouse on 
trying to prevent kids from dropping out of school, I think there is 
the kind of information there that people can get their arms 
around. 

I have to tell you that at the state level we moved offices several 
years ago and we had box after box after box of state dropout ini- 
tiatives. There was not one piece of evidence tied or — in any of 
those boxes. Nothing had been evaluated. 

So it is just so critically important, and it doesn’t always get 
done. 

Ms. Long. I think the goals of the What Works Clearinghouse 
are very admirable. Education is a very crowded space and there 
are many opinions and there are many organizations that are re- 
leasing different reports. 

And because much of the research is very complex, the average 
person oftentimes cannot understand it and so lES has stepped in 
as a federal role to say, “We will be an objective, unbiased source. 
We will help to translate what all this complex work means.” 

And that is where the What Works Clearinghouse steps in, look- 
ing at studies to say, “People are making these grand statements. 
Can you actually believe these results?” 

To walk away and say, “This program does or does not work.” 
Knowing that it does not work is also very important. 

But the What Works Clearinghouse, I would say, has many op- 
portunities for improvement in how it interacts with the field, from 
the way it does its studies to making sure that people in the field 
know that it exists. There have been recent changes even to its 
Web site so that when someone goes it is organized in terms of fre- 
quently asked questions — I want to know about how to help my 
preschooler — and starting to organize and revamp the site in that 
way. 

So it is definitely an area of continuous improvement, but the 
goal of trying to give an objective, clear, easy to understand re- 
source is absolutely something that we need. 

Mr. Scott of Virginia. Once you know what works how do you 
get — do you have a training mechanism so teachers can actually 
get trained in what works or is it just sit there, they find out what 
works but don’t know how to do it? 

Mr. Kemple. I think the mechanisms for translating the evidence 
into practice are — in many ways are still wanting. I think from a 
policymaking perspective and from the perspective of trying to de- 
cide how to allocate scarce resources, I think one critical lever may 
very well be ensuring that if a state or a district wants to use fed- 
eral resources or a district wants to use state resources, or if a 
school has access to some flexible resources they should be obli- 
gated to demonstrate that the way that they would like to use 
those resources has appropriate and rigorous evidence behind its 
effectiveness, and that they have a way of being able to link with 
the people who either have produced that evidence or started to 
translate it into guidelines — that they can use to implement their 
programs 
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Mr. Scott of Virginia. Well, but how does — how does a teacher 
get trained in what works? I mean, he or she is sitting in the class- 
room, you read this research works, how do you learn how to do 
it? 

Ms. Long. I think that is where the practice guides come in, be- 
cause that is taking of research and putting it into the pieces of 
the day-to-day job of a teacher, now what do you do with this infor- 
mation? How do you actually apply it in your classroom? And so 
that is another translation function that lES has taken upon itself. 

Mr. Scott of Virginia. And is it working? 

Ms. Christie? 

Ms. Christie. If I could just make one suggestion that I don’t 
hear talked about at all, and that is if every time there was a new 
piece that was put into the What Works Clearinghouse or a new 
conclusion drawn, if that could be pushed out to school boards 
across this country, it is school boards, then, that help decide how 
professional development is delivered, how teachers collaborate 
within the districts, what they should be looking at. 

And lES already has the contact information for every district. 
You wouldn’t have to have all the boards. The districts then could 
relay that to the 

Mr. Scott of Virginia. My time is almost expired. I wanted to 
get in another question about whether or not there is research on 
nonverbal communications, which can be very important in how 
children react to education. 

Mr. Kemple. Not something I am very familiar with. 

Mr. Scott of Virginia. You are not? 

Mr. Kemple. Not, no. 

Ms. Christie. I am not sure what you are asking. 

Mr. Scott of Virginia. Nonverbal communications and teacher 
interacting with the student, conveying caring or not caring and 
the children reacting to that. Is there any research on that? 

Ms. Christie. I am not familiar with the research on that but 
I am familiar with the tools that lES has to quickly find out who 
else knows or done any research on that. 

Mr. Scott of Virginia. Thank you, Mr. Chairman. 

Mr. Rokita. I thank the gentleman. 

Mr. Holt is recognized for 5 minutes. 

Mr. Holt. Thank you. 

I thank the witnesses. 

This is encouraging. I am hearing a — what I had hoped to hear 
about lES. 

I will begin by saying we badly need rigorous research. We have 
been hampered for, well, decades — maybe forever — because every 
policymaker, every school board member, policymaker on the state 
or federal level was a student and, therefore, is an expert on edu- 
cation. And so we end up with the same old, with frequent overlays 
of short-lived fads. And so I hope that we can make the most of 
this rigorous research. 

The proposal came from Mr. Walberg on the other side that we 
devolve this to the states or turn it over to the private sector. Let 
me ask two specific questions: Are most states doing rigorous, rel- 
evant research? Does anybody 
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Ms. Long. My sense is the answer is no because they don’t have 
the capacity to do it. 

Mr. Holt. Okay. Is the private sector doing sufficient studies of 
rigorous, unbiased research that teachers or policymakers can 
trust? 

Mr. Kemple. On their own initiative I would say very few, if any. 

Mr. Holt. So the conclusion I draw from that is we need this. 
So let me ask the next question: Do we need more of this? Are we 
close to saturation in producing the kinds of rigorous studies that 
we need to wisely spend many tens of billions of dollars every year 
in actually — well, in education? 

Ms. Long. No. Not at all. 

Monitoring the applications for research proposals over time to 
lES, the quality of them has increased dramatically over time. And 
unfortunately, many of them are not funded, and during the last 
year there were a number of very highly rated research proposals 
through the peer review process that unfortunately were not able 
to be funded. 

So there are many great 

Mr. Holt. More than were funded? Were a minority of the well- 
reviewed proposals funded? 

Ms. Long. That is correct. That is correct. 

Mr. Holt. Okay. Does the lES review the — have there been stud- 
ies within the lES of the application of former studies? Does the 
lES review the application over the years or the follow up that oc- 
curs on those studies? 

Ms. Long. There has been a lot of attention. It has looked back 
and it is hard to find out what impact those have had, and so this 
year there is actually a competition to establish a center of knowl- 
edge utilization just to address that issue as well as requiring fund- 
ed researchers to have dissemination plans and communication 
with the field so that their research results are used by those prac- 
titioners. 

Mr. Holt. Changing the subject, one of the things we are asked 
to consider reauthorizing would be the state longitudinal data sys- 
tems. Are any of the witnesses expert in the data systems? My spe- 
cific question is, are we learning from the good data systems things 
that are useful in the classroom? 

Mr. Kemple. I think we are just beginning that process. In New 
York City we have formed an explicit research alliance that works 
closely with the New York City public school system and have been 
able to acquire its entire administrative records database, along the 
same lines as what the state longitudinal data systems are sup- 
posed to be doing across the country. 

It has turned out to be an incredibly valuable tool both for re- 
search and for policy and practice. I think to the degree that we 
can continue to help states both create these systems and then 
have them link up with the capacity to make use of those for re- 
search purposes, I think we will quickly see mechanisms that will 
allow that work to penetrate into schools and classrooms. 

Mr. Holt. And then I guess this is specifically for Dr. Kemple: 
How can research be embedded in the programs that are author- 
ized and funded? How good a job are we doing at it? How could we 
make that happen? 
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Mr. Kemple. Again, I think this is something that has a long 
way to go, I think in two ways — one, particularly when there are 
hard decisions about how to make use of resources that can be used 
for innovation. I think any use of resources for innovation, be they 
from the federal, state, or local level, should be accompanied either 
by strong evidence that this innovation has the capacity to change 
teaching and learning in schools for the better; or B, if there is 
lacking evidence, that there is a requirement that the participants 
or the recipients of the funding be willing to participate in a rig- 
orous research study to establish whether or not those resources 
are paying off in better teaching and learning. 

Mr. Holt. Thank you. 

Mr. Rokita. I thank the gentleman. 

Mr. Hinojosa, you are recognized for 5 minutes. 

Mr. Hinojosa. Thank you, Mr. Chairman. I thank you all for in- 
viting this wonderful panel to talk to us about improving teaching 
and learning in classrooms across our country. 

I want to say that I am a strong supporter of the regional edu- 
cational laboratories and the comprehensive center programs. My 
first question is going to go to Dr. Long about the new emphasis 
at the Institute of Education Sciences, you call lES, on knowledge 
utilization, which I think is just a fancy way of saying we want to 
ensure that educators know about the research on what works. 

Therefore, can you tell us about the work at lES Sciences in this 
area with early childhood development? 

Ms. Long. With early childhood development specifically? Unfor- 
tunately, I can’t speak specifically to that area. I can say, generally 
the issue has been how do we translate our research, research that 
might be on early childhood education — while also trying to under- 
stand from the practitioners’ view, what are the questions that 
they have, what are their pressing needs. You have to understand 
how they do their work so how they might actually change practice 
with the research results, so that it can’t be just one-directional 
from lES. What lES needs to do and is trying to do is build a two- 
way street so we have communication from the people who work 
with our young children of what they need, what are the pressing 
questions, and how they might — how they do their work and then 
how the research might be able to inform that. 

Mr. Hinojosa. I am going to ask another question and probably 
come back to you. Dr. Long. 

Ms. Christie, you spoke about your learning the importance of 
education from cradle to age 3 years of age, and you used your chil- 
dren that you stayed home and educated them at home. Why do 
you think that state and federal elected officials do not consider a 
high priority to invest in early education programs? 

And the reason I ask you that question is that years ago Buck 
McKeon was chairman of our Subcommittee on Higher Ed and took 
a codel to China. We were trying to find out why it was that they 
would usually beat the United States on competition — international 
scholastic competition. And one of the people who answered our 
question was an older gentleman — a professor — and he said it was 
a very simple answer. 

The formula he said was early reading plus writing equals suc- 
cess in school, and that they started reading to the children the 
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moment they were born. So they read to them in the cradle and 
then they already knew how to read by the time they were 3 years 
old and pecking on a computer at age 4 to tell us what they read. 
It was that simple. 

So you tell me, was it research that you read that made you be- 
lieve that you wanted to stay home and spend the time you did 
with your children from the time they were born on? 

Ms. Christie. When I was in my preparation program for being 
educated as a teacher, yes, that was part of the program. I do be- 
lieve state policymakers are very compelled to address those early 
years. Sometimes it is simply a fiscal issue, and I think we are see- 
ing the investments go up. 

We track state of the state addresses by all the governors and 
I can tell you, it was a prominent part of a number of governors 
this year and that is not unusual. We do see that. So they are 
very — they are not uninvolved in it. They are very, very interested 
and most of the time it is a fiscal issue. 

Mr. Hinojosa. Dr. Long, do you have research that shows that 
it is working, where they invest in the regions of the country in the 
cradle to age 3 and 4 on early childhood development? 

Ms. Long. There is certainly a research basis for those early in- 
vestments have long-term impacts. There have even been studies 
that looked at children in preschool and then followed them years 
and years later to see that they were doing better in high school 
and better in college. So there is that research basis. 

And I would say in a short amount of time, as we have started 
to collect this information, fund additional information, and put it 
in a central location so we can figure out how all the different 
pieces of the puzzle fit together, we are starting to come out with 
some strong conclusions about what does work, although there is 
so much more that we don’t understand and so many things we 
have found that don’t work, and so we need alternatives. 

Mr. Hinojosa. Thank you, Mr. Chairman. I yield back. 

Mr. Rokita. I thank the g:entleman. 

I just have a couple questions. 

Again, I want to thank the witnesses. I appreciate all your testi- 
mony. Let me start with Mr. Scott. 

Your testimony stated, I believe, that the performance measures 
that lES uses do not reflect current programs, so can you provide 
some specific examples of this? 

Mr. Scott. Mr. Chairman, I do think it is important to note that 
when you look across any number of the current performance 
measures lES has in place, they are actually doing quite well in 
many areas. A couple areas that we have identified where we be- 
lieve there needs to be more transparency and accountability in- 
cludes the RELs as well as some of the new research grant pro- 
grams that lES has not yet developed performance measures for 
those activities. 

Mr. Rokita. And what can we be doing better to — or lES be 
doing better to accurately evaluate its programs? Is that the same 
thing? 

Mr. Scott. I think first of all, involve policymakers and practi- 
tioners in the research agenda setting; develop relevant, timely, up- 
to-date performance measures; have the information necessary to 
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evaluate those activities; and then hold those activities account- 
able. 

Mr. Rokita. What was the second to the last one? 

Mr. Scott. Develop the measures, involve stakeholders in the re- 
search agenda setting 

Mr. Rokita. What would a couple examples of performance 
measures be in your mind? 

Mr. Scott. You know, for example, identifying feedback opportu- 
nities to involve stakeholders, measuring how you are doing in re- 
lation to what the stakeholders’ needs are. They do have a number 
of measures related to the What Works Clearinghouse, in terms of 
the level — the number of interventions that have been supported by 
their activities. 

And so I do think, you know, they have made progress in this 
area, but particularly as it relates to the regional educational labs 
and some of the new grant programs, we do believe it is critically 
important that you establish performance measures in those 
areas — particularly the RELs. That is a significant investment on 
the part of lES, and so to not have public reporting and not have 
public accountability around that activity we believe is a key area 
for improvement. 

Mr. Rokita. And again, what would — in your mind, what would 
that public reporting look like? 

Mr. Scott. As again, you know, having key indicators, perform- 
ance measures, in terms what the expectations are for the RELs, 
but then having feedback 

Mr. Rokita. What would that be? I am trying to get you to be 
specific and quantitative. 

Mr. Scott. Quantitative. Could be, once again, meeting the 
needs — having a feedback loop in terms of meeting the needs of the 
stakeholders, having clear expectations in terms of how you expect 
the RELs to be engaged with the research alliance, and then hold 
them accountable for that cooperation. 

Mr. Rokita. Okay. I am going to leave you alone. 

[Laughter.] 

Mr. Scott. I don’t want to get too prescriptive here. I mean, one 
of the things we 

Mr. Rokita. Okay. Fair enough. 

Mr. Scott [continuing]. Do believe is having both quantitative 
and qualitative measures in place. 

Mr. Rokita. There we go. You got 

Mr. Scott. Customer service satisfaction. That is the word I was 
looking for when I was talking about getting feedback. I mean, the 
National Center for Education Statistics does a really good job of 
that, and when we talked to the stakeholders that is one of the 
things they pointed out is that a lot of the projects and the prod- 
ucts produced by NCES are really useful, and there are clearly op- 
portunities for lES to do that across more of its activities is to get 
that direct customer satisfaction survey information and use it di- 
rectly. 

Mr. Rokita. Okay. 

Dr. Long, your testimony — I conclude lES has a broad mission to 
provide useful information to many different audiences. In times of 
limited resources, though, and this was touched on a little bit ear- 
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Her, in your opinion, who can benefit most from lES products? Re- 
member, we are broke. 

Ms. Long. Yes. There are many audiences. I think policymakers 
absolutely need the information that is coming from lES to make 
better decisions what to do with limited resources, what works, and 
for what cost. 

I think researchers absolutely sending signals to them about how 
they should spend their time. I don’t think lES needs to do every- 
thing; it needs to leverage the field. And by using the signals, the 
incentives, its convening power to take the researchers out in the 
field, the organizations, whether they be states or school boards, to 
get that information out to them. 

I think it also does have a translation responsibility to the field 
to make clear what do you do with this research information, but 
then again, working in concert with other organizations and asso- 
ciations to get the information out there. 

Mr. Rokita. And for what audience? Eor policymakers, you say, 
and for who else? 

Ms. Long. I said for policymakers, I said education researchers 
should help direct where they are putting their efforts. I said in 
terms of the field — so teachers, principals, superintendents — the 
translation function of lES is very important. But again, to get that 
out I think it is working in concert with organizations and associa- 
tions that are in the field — not for them to necessarily do every- 
thing else, but they have the unique position of being completely 
objective, having convening power like no one else does, and so 
they can send signals, incentives, bring people together in a very 
different way than any other organization. 

Mr. Rokita. Okay. Fair enough. 

And then in the 30 seconds I have left, if the two remaining wit- 
nesses would like to respond to either of my questions, you are wel- 
come to. 

Dr. Kemple, in about 10 seconds? 

Mr. Kemple. Yes. I think the Congress and the U.S. Department 
of Education I think would be the primary beneficiaries of IBS’s 
work, both on the quality and the quantity of work that gets pro- 
duced. And in terms of answering questions about what — not just 
about effective practices but also about ineffective practices, I think 
it is the responsibility to the public to be able to 

Mr. Rokita. Thank you. 

Mr. Kemple [continuing]. Invest in learning about whether 

Mr. Rokita. Appreciate that. 

Ms. Christie? 

Ms. Christie. I would like to suggest that folks look at the gov- 
ernment performance accountability system in Washington, go to 
their site, look at the kinds of metrics they report on on their site. 
It is very impressive, and I wish more folks, both state and federal, 
would do the similar thing. 

Mr. Rokita. Okay. Thank you. 

Again, I thank the witnesses and now I yield to the Ranking 
Member Miller for his closing remarks. 

Mr. Miller. Thank you very much, and I want to thank the 
chairman of the subcommittee, and the Chairman, for holding this 
hearing. I think it is very important. 
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You know, I think that the testimony is pretty clear that we have 
developed a much better system than we had before, and the ques- 
tion is, how do you hold on to that and how do you maintain the 
integrity of that system and sort of grow the confidence of the prac- 
titioners, if you will, from governors on down all the way to the 
classroom. 

And, Mr. Scott, how do you get into a situation, as you point out 
on page six here, on this question of the peer review process start- 
ing to stretch itself out and growing from 117 days in 2011 to 175 
days in 2012? I mean, this can be the death of an organization. 

Mr. Scott. And I think that is 

Mr. Miller. You know, we are in a dynamic system where, at 
our level in our districts people think about this in 9-month seg- 
ments, and what is new has got to show up on a fairly regular 
basis, I mean, and coincide with the needs of the users, which are, 
dictated by the school year if nothing else. 

Mr. Scott. Well, I think that issue sort of points to two things 
that we talk about in our statement, one being the importance of 
sort of balancing the rigorous research with ensuring it is timely, 
and useful, and relevant. And the second issue is having lES use — 
lES gathers a lot of information about a lot of things going on 
under its — or in its purview, but we have found that at times there 
are gaps in how they use that information to oversee the research, 
and the issue of the peer review is a perfect example of that. 

It is not clear to us that they were really aware of this until we 
started asking some questions about what was going on with the 
timing here, and now they are paying attention to it. Our point to 
lES would be there are other processes that you have in place that 
you should be constantly monitoring to ensure things don’t spin out 
of control or start to grow and expand in a way that negatively im- 
pacts the end users of the information and the research. 

Mr. Miller. So you think this can, in fact, be corrected, in terms 
of getting the peer review condensed — I mean, if people have 
signed on to participate in that process, maybe they have signed 
on to do 1,000 of these, I don’t know, but at some point you have 
got to know what your contractors — what the capacity is to partici- 
pate. You may want them for their name, for their specialties and 
their expertise, but if they don’t allow the time so they can fully 
participate you have got to move on and find, I think, somebody 
else at some point. 

Mr. Scott. It is important for lES to have the right information 
to monitor the performance and then to take the necessary correc- 
tive actions to ensure that that peer review process doesn’t con- 
tinue to grow and negatively impact its ability to provide the re- 
search. 

Mr. Miller. Can you just put in, — quickly, your concerns about 
the regional education labs? You come across that in your testi- 
mony — 

Mr. Scott. If you look historically at some of the challenges 
around the regional education labs, we continue to be concerned 
that there are not clear performance measures for the RELs, that 
it is not publicly available. lES is collecting some information. lES 
also has an ongoing evaluation of the previous cohort of the RELs 
that is due to be completed at some point here in the near future. 
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We do think it is important, though, for accountability reasons 
that there be some public accounting for those RELs, that they 
have clear performance measures and indicators, and then that 
lES take the necessary action to hold them accountable for their 
performance. 

Mr. Miller. I mean, this is — at least as it has been presented — 
this is supposed to be some high performance operation, and the 
question that you are raising to some extent is whether or not that, 
in fact, you are getting a high performance operation in some of the 
regions 

Mr. Scott. Well, we have heard comments from certain stake- 
holders that certain RELS are more productive than others, the 
relevance of the research that certain RELs produce is more rel- 
evant than others. I think the question we have for lES is, you 
know, at what point are we going to have more public accounting 
for the performance of the RELs? That is where you start — public 
accounting for their performance. And then you can make the nec- 
essary decisions after then how to move forward. 

Mr. Miller. Thank you. 

Thank you, Mr. Chairman. 

Mr. Rokita. I thank the ranking member. 

Let me again and finally thank the witnesses for your testimony. 
It was very enlightening. Very much appreciate it. 

I think what we learned is that the quality of education research 
has greatly improved in the last decade, and for that we are thank- 
ful. And we are thankful in particular for your leadership in the 
field. 

However, like most things in life, there are places that we can 
look to strengthen the evaluation and performance requirements 
for lES programs and ensure that the research coming out of it is 
rigorous, relevant, and useful to education practitioners. And I 
think you saw that in the questioning by several members this 
morning. I think that ought to be our next goal and a continuation 
of the work that we have done. 

So I look forward to working with my colleagues to reauthorize 
the Education Sciences Reform Act in a positive way that recog- 
nizes the fiscal condition we are in, how to leverage not only the 
resources of lES but its partners at the federal and state level, and 
certainly the leadership of each of the witnesses here today. 

And with that, seeing no further business before this committee, 
this hearing is adjourned. 

[Whereupon, at 11:27 a.m., the committee was adjourned.] 
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